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ABSTRACT 

Legislation has largely treated planning and 
evaluation in vocational education separately. There has been a 
history of increasing requirements in planning and evaluation and of 
diffusing responsibility. In addition, existing policy regarding 
vocational education's dominant mission shows a general lack of 
clarity. Such factors inhibit the linking of planning and evaluation. 
The planning literature shows that vocational educators have tended 
to emphasize certain approaches to and uses of planning. Although 
planning theorists have developed multiple approaches that might 
address many of the contemporary problems facing vocational 
education, these approaches have had very limited use. A conclusion 
is that alternatives to the dominant "rational-comprehensive" 
approach to planning can facilitate the useful linking of planning 
and evaluation. The evaluation literature reveals a gradual evolution 
of the scope and methodology of evaluation in response to political 
and organizational contexts. The theme of evaluation utilization 
emerges as critical to the linking process. A hypothesis is that the 
primary benefit of using evaluation will be realized through its 
capacity to institutionalize organizational learning. These multiple 
streams of literature can be brought together in a proposal for a new 
metaphor for linking in which planning and evaluation are seen as 
mutually inclusive rather than exclusive. (Contains 102 references.) 
(YLB) 
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PREFACE 



As the result of federal legislation, evaluation and planning in vocational education 
have received increased emphasis since the mid-1970s. States have responded by 
developing various models, systems, and procedures for the generation, collection, and 
analysis of labor market, enrollment, programmatic, and follow-up data. Most states have 
some approach to planning and evaluation at the state level. However, these approaches 
vary widely in scope and sophistication and usually do not support each other to a desirable 
extent. 

The project Unking Planning and Evaluation in Vocational and Technical Education 
includes four major components. The first component resulted in a concept paper that 
presented a way of thinking about linkages and framed a series of questions to guide 
further investigation of the interaction between planning and evaluation. The second 
component involved a comprehensive review of planning and evaluation literature and is 
the subject of the present document. In the third component, all states and territories were 
surveyed to determine current practices related to planning and evaluation. Based on the 
survey results, three states were chosen for in-depth, on-site interviews. The final 
component of the project was a document designed to assist vocational planning and 
evaluation personnel in the development and implementation of coordinated planning and 
evaluation systems. 

The purpose of this paper is to present a synopsis of the literature reviewed in the 
second phase of the project and to propose for consideration a new metaphor for linking 
based on that literature. The literature review is not simply a commentary or a synopsis but 
an exercise in critical reflection. As such, it not only attends to the literature but, more 
significantly, attempts to transcend the individual pieces of work by relating them in a 
conceptual map. This map is intended to help in applying the various planning and 
evaluation approaches to the unique characteristics and problems of vocational education. 
The final section of this document brings together several "streams" from the planning and 
evaluation literature and proposes a new metaphor for linking planning and evaluation 
within the larger policy design — the policy implementation continuum. 



The literature search was conducted in three phases: 

1 . A manual search through the Social Science Index and the Public Affairs 
Information Service Index for publications on planning and evaluation and 
their linkages. The search covered literature produced between 1978 and 
1988. 

2 . A broad Educational Resources Information Center (ERIC) search covering 
the period from 1978 to 1988 to locate publications regarding planning and 
evaluation and their linkages in vocational education. Three small searches 
were then run to isolate more specific materials using synonymous terms 
and alternative descriptors. Secondary searches were conducted for 
documents cited in primary references. 

3 . The searches completed in item two above were repeated near the close of 
the project to provide up-to-date material in finalizing this review. 

The first two searches identified materials from a variety of areas in public 
administration, education, and other social sciences. None of the searches, however, 
identified research which specifically addressed the issue of linking planning and evaluation 
in vocational education. Lacking examples to provide insight into practices that facilitate or 
inhibit linking, it was decided that the available materials from related fields would be 
synthesized. Once the apparently relevant research and strategies from these fields were 
located the material was critically reviewed for its applicability to the vocational education 
setting. 

The results of the review are organized into the four sections of this paper. The 
first section reviews key ideas developed in the earlier concept paper and presents an 
overview of the statutory bases for planning and evaluation in vocational education. The 
definition and theoretical bases for planning arc presented in the second section. The scope 
and methodology of evaluation, as well as political perspectives and their impact on 
utilization of evaluation, are reviewed in the third section. The final section looks at the 
interaction between planning and evaluation, examines the linkages and their constraints, 
and proposes a new metaphor for linking planning and evaluation in vocational education. 
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EXECUTIVE SUMMARY 



This document is one of several items produced as part of a National Center for 
Research in Vocational Education (NCRVE) project titled Linking Planning and Evaluation 
in Vocational and Technical Education. A concept paper, Linking Planning and Evaluation 
in Vocational Education: Concepts and Issues, has also been prepared (Asche, Strickland, 
& Elson, 1992). 

A review and synthesis document of this type necessarily omits literature of 
potential worth to the reader in an attempt to focus on that which "fits" within the frame of 
reference of the reviewers. In the present case, the authors operated with the assumptions 
that inadequate linkages between planning and evaluation exist within vocational education, 
that improved linkage would result in more effective reconciliation of policy design and 
policy action, and that there is a base of theory and research from which vocational 
education can benefit. 

The review begins by providing a brief summary of the statutory bases for both 
planning and evaluation in vocational education. It was found that these two functions 
have largely been treated separately in legislation. There has been a history of increasing 
requirements in both planning and evaluation and of diffusing responsibility. In addition, 
there is a general lack of clarity in existing policy regarding vocational education's 
dominant mission. Such factors inhibit the linking of planning and evaluation. Yet, 
planning and evaluation can be more appropriately addressed as a result of their linking. 

The review of planning literature revealed that vocational educators have tended to 
emphasize certain approaches to and uses of planning. While planning theorists have 
developed multiple approaches that might address many of the contemporary problems 
facing vocational education, these approaches have had very limited use. It was concluded 
that alternatives to the dominant "rational-comprehensive" approach to planning can 
facilitate the useful linking of planning and evaluation. 

The review of evaluation literature revealed a gradual evolution of the scope and 
methodology of evaluation in response to evaluation's political and organizational contexts. 
During the course of the present research, the theme of evaluation utilization (in particular, 
strategic utilization) emerged as critical to the linking process. It was hypothesized that the 
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primary benefit of utilizing evaluation will be realized through its capacity to institutionalize 
organizational learning. 

The multiple streams of literature related to planning, evaluation, evaluation 
utilization, strategic planning, and organizational learning were brought together in the final 
section of this review in an attempt to examine both the facilitators and barriers to linking 
planning and evaluation. A new metaphor for linking is proposed in which planning and 
evaluation are seen as mutually inclusive rather than exclusive. The pragmatic connection 
of planning and evaluation becomes, as stated by Hoch (1984), a "process of practical 
inquiry in which problematic experience stimulates the critical assessment of alternative 
solutions in a context of democratic participation and debate" (p. 340). 



iv 
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OVERVIEW OF PLANNING AND EVALUATION 



Two common laments in the literature on planning and evaluation concern plans that 
do not get implemented and evaluation findings thai do not get used. Planning and 
evaluation are generally treated as distinct functions. The separation of the two is evident 
not only in terms of various research methods but also in terms of the various professional 
communities and the different departments and personnel within an agency. The earlier 
concepts and issues paper from this project proposed that this is a counterproductive state 
of affairs that resides within an even larger cycle of policy design and policy 
implementation. The complaints about nonimplementation or distortion of policy intent on 
the one hand and charges of naive or ill-informed policy on the other parallel the laments in 
the planning and evaluation literature. 

Planning and evaluation in vocational education have both practical and legislative 
bases. The formal processes have been strongly shaped by legislative requirements, but 
the practical planning and evaluation needs of states and localities also seem to be served by 
less formal procedures. These activities take place at a level subordinate to the more visible 
"plans" and "evaluations" produced to satisfy legislative requirements (Strickland & Asche, 
1987). 

The earlier concepts and issues paper proposed the notion — or more accurately, the 
hypothesis — that planning and evaluation is ideally a single process — an ongoing, regular 
dynamic between two interrelated functions. Such a notion assumes that planning and 
evaluation activities ultimately have the same overall mission and not simply different but 
complimentary missions. The primary mission for these activities was the reconciling of 
policy design and policy action. Practitioners often understand this relationship intuitively, 
but there are many barriers to realizing the ideal of a single planning and evaluation 
dynamic. 

Operational Definitions 

A beginning point in the investigation of linking planning and evaluation is the 
definition of the two terms. There is certainly no shortage of definitions for planning and 
evaluation: There is some disagreement among authors regarding terms and the 



classification of type, but there is general agreement across the many definitions regarding 
substance. The following definitions are those which were developed in the earlier 
concepts and issues paper and which have proven to be most helpful to the present review 
(Asche, Strickland, & Elson, 1992). 

Planning 

Hudson and Davis (1976) included in their definition of planning the development 
and statement of goals; the determination of policy and program alternatives; the assessment 
of costs and resources; the evaluation of outcomes or effects; and the monitoring of 
allocations, decisions, and implementation activity. Regarding this definition, it is useful to 
think of planning as involving at least three major functions: (1) the setting and 
modification of goals and purpose; (2) the systematic determination of policy and program 
alternatives and of priorities; and (3) the use of appropriate needs, cost, resource, output, 
and outcome data to monitor programs and provide feedback to the planning process on a 
continuing basis. Asche (1989) labeled similar functions in the vocational education 
context as (1) substantive planning — setting the directions and mission for vocational 
education; (2) management planning — translating goals into priorities and allocating 
resources; and (3) operational planning — operationalizing goals and priorities as programs 
and services and providing feedback. Both of these definitions serve to illustrate the 
"fuzziness" of the line between the traditional purposes of planning and evaluation. The 
second section of this document expands on the concept of planning. 

Evaluation 

Scriven (1973) provided one of the most concise definitions of evaluation: to 
determine the worth or merit of whatever is being evaluated. Brophy, Groteleuschen, and 
Gooler (cited in Worthen & Sanders, 1987, p. 6) categorized three major purposes for 
evaluation: (1) planning procedures, programs, and/or products; (2) improving existing 
procedures, programs, and/or products; and (3) justifying (or not justifying) existing or 
planned procedures, programs, and/or products. There is general agreement that evaluation 
may serve either a formative purpose (to improve a procedure, program, or product while it 
is in operation) or a swnmative purpose (to evaluate the procedure, program, or product in 
terms of continuance or discontinuance). 



These views of evaluation include functions often ascribed to planning, just as 
certain planning functions might easily be considered pan of evaluation. Without an 
informed awareness of the overlap, the potential for conflict and dysfunction is apparent 
(Asche et al M 1992). The third section is devoted to a discussion of the scope and 
methodology of evaluation. 



The Statutory Basis of Planning in Vocational Education 

The genesis of planning in vocational education was the Smith-Hughes Act of 1917 
(U.S. 64th Congress, 1917) — particularly sections five, eight, and ten. The act provided 
the "legal basis on which both state and federal field representatives could make 
administrative decisions' 1 (Swanson, 1966, p. 68). In general, states locked into the 
statements of intent derived from the requirements of the act as a model for planning, 
McDonnell and Grubb (1991) noted that "as the federal government expanded its own 
programs and more precisely targeted aid, it shaped state policy and local service delivery" 
(p. 2). As a result, state plans took on the character of "compliance documents" to ensure 
the continuity of state programs throughout changing administrations. However, planning 
for compliance failed to initiate planning for change. Barlow (1990) suggested that policy 
decisions for vocational education are influenced by d inges in federal legislation and 
societal views: 

Policy serves to guide strategies for implementing educational programming 
at the local, state, and federal levels. In the lane/ case, it seems clear that 
federal legislation becomes the same as federal policy. Moreover, in spite 
of the constitutional view that education is a right of die states, when states 
accept federal monies for vocational education, the expenditure of those 
dollars must be in accordance with federal guidelines, and thus federal 
policy becomes the policy of the state. Similarly, state policy strongly 
influences local policy in many areas, (p. 25) 

The compliance character of planning became more entrenched in the period from 
1963 to 1975 with the passage of the Vocational Education Act of 1963 (U.S. 88th 
Congress, 1963) and the subsequent Vocational Education Amendments of 1968 (U.S. 
90th Congress, 1968) and Education Amendments of 1972 (U.S. 92nd Congress, 1972). 
Under the Vocational Education Act of 1963 , states developed five-year plans (updated 
yearly) and annual plans. To develop these plans, states initiated more regular data 
collection procedures, evaluated the previous year's activities, determined goals for the 
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following year, and produced a written plan (Moss, 1983). After state advisory councils 
were established by the 1968 Amendments, all state plans required approval of the state 
vocational education councils. In addition, the state plans indicated how vocational funds 
were to be distributed to groups with special needs. 

The Education Amendments of 1976 (U.S. 94th Congress, 1976) reiterated most of 
the planning requirements of the 1963 act In addition, the 1976 Amendments emphasized 

1. the importance of serving the disadvantaged, the handicapped, 
women, and other minority groups; 

2. the concern for improving the efficiency of vocational education and 
for coordinating it with other federal and state occupational training 
programs; 

3. the review and preparation of recommendations for long-range 
plans; and 

4. the creation of national and state occupational information 
coordination committees (NOICCs and SOICCs) to develop and 
monitor management information systems. (Moss, 1983) 

David (1981) reported that prior to the 1976 Amendments, planning documents 
were very compliance oriented. By 1980, David (1981) noted, "The States were 
complying with the procedural requirements of the statutory provisions. But whether this 
meant that the desired kind of systematic planning of vocational education programs had 

been generally achieved is another question State officials were still reporting in 1980 

that the State plans were being produced for compliance purposes" (p. K-4). 

In 1984, the Carl D. Perkins Vocational Education Act (U.S. 98th Congress, 1984) 
made many changes in vocational education. Specifically related to planning was the 
requirement for two-year plans that placed greater emphasis on students with special needs. 



The Statutory Basis of Evaluation in Vocational Education 



( 



State advisory committees were established by the Vocational Education Act of 
1963 (U.S. 88th Congress, 1963). One of the responsibilities of these advisory 
committees was the evaluation of vocational programs within their respective state 
(Wentling, 1980). Before that time, very litde evaluative action took place in state or other 
educational agencies. However, Wentling (1980) reported that 

most of the state advisory committees did not assume the responsibility for 
evaluation as the legislation had intended. It as not until the passage of the 
Elementary and Secondary Education Act of 1965 [ESEA] that evaluation 
really became a prime concern of educators at the state and local level. In 
essence, ESEA legislation required that each project conducted under Title I 
and m possess a specific evaluation component — to include a plan for the 
evaluation of process and product, (p. 7) 

The Vocational Education Amendments of 1968 (U.S. 90th Congress, 1968) re- 
emphasized the requirements of the 1963 act. For instance, it parceled out the evaluation 
activities to the national council and the state advisory councils. These responsibilities 
included the following: 

1 . "Review the administration and operation of vocational education 
programs, including the effectiveness of such programs in meeting 
the purpose for which they were established" (Section 
104(a)(2)(B)). 

2. "Conduct independent evaluations of programs carried out under" 
this act (Section 104(a)(2)(C)). 

3. "The National Council shall review the possible duplication of 
programs at the postsecondary and adult levels within geographic 
areas " (Section 104(a)(5)). 

4. The State advisory council shall, "evaluate vocational education 
programs, services, and activities under" this act (Section 
104(b)(1)(B)). 

The Educational Amendments of 1976 (U.S. 94th Congress, 1976) expanded the 
mandate for evaluation efforts, including various new components: state employment 
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needs evaluation; program review; fiscal audit; entry-level jobs skills training programs 
evaluation; and a uniform, national vocational education data reporting and accounting 
system. The responsibility for these efforts was dispersed among state advisory councils, 
state boards, the Commissioner of Education, the Bureau of Occupational and Adult 
Education, the National Center for Educational Statistics, the National Occupational 
Information Coordinating Committee, the National Advisory Council on Vocational 
Education (NACVE), the National Institute of Education (NDE), and state manpower 
services councils. 

David (1981) summarized the evaluation requirements of the 1976 Amendments 
when he wrote that "the new provisions affecting evaluation constituted a key feature of 
Federal legislation. For the first time, Federal and State Governments were required to 
conduct systematic program evaluations" (p. IX-3), According to David (1981), 
evaluations conducted in most slates before the 1976 Amendments were informal. After 
reviewing three NIE studies of state evaluation practices conducted from 1978 through 
1981, David (1981) concluded that "only one of the four approaches to evaluating the 
effectiveness of programs specified in the regulations — that dealing with planning and 
operational processes — has the potential to prove useful for the purposes of improving 
programs and decision making on program offerings, at least in the immediate future" (p. 
IV-22). 

The Carl D. Perkins Vocational Education Act (U.S. 98th Congress, 1984) re- 
emphasized the stipulations of the 1976 Amendments for program evaluation in individual 
states* The net effect of this and previous statutory requirements has been to reinforce the 
compliance activity that marks the current general status of state plans (Strickland & Asche, 
1987, p. 13). 

Recent Developments in Vocational Education Legislation 

The Carl D. Perkins Vocational and Applied Technology Education Act of 1990 
(U.S. 101st Congress, 1990) continued the emphasis on statutory planning and evaluation: 

Congress continues its reliance on inducements to influence who is served 
in vocational education. At the same time, expanded definitions of the 
policy problem by the business community and others have led Congress to 
move beyond its traditional emphasis on policy targets; recent legislation 



now specifies the nature of the services to be provided and requires that the 
outcomes of those services be publicly reported. (McDonnell & Grubb, 
1991) 

The act enumerates a long list of assurances required of a state wishing to receive 
funds. Evaluation is required on two new and significant fronts. First, a state must 
conduct an assessment (Section 1 16) of all vocational programs prior to the preparation of 
the state plan. The second evaluation is based on the "core standards and measures of 
performance" (Section 115). The performance measures and standards are mandated in 
specific areas. However, the states are given some latitude in the exact statement of the 
performance measures and standards and the level at which the standards are set The act 
continues the tradition of past acts and amendments of legislating the procedures to follow 
in evaluating vocational programs. Section 1 17 requires an annual evaluation of "the 
effectiveness of the program." The section goes on to mandate the development of a "local 
program improvement plan" and a "state and local joint plan." These areas of assessment 
require a focus on student outcomes. The act increases the emphasis on traditional labor 
market, training capacity, and special populations services. These requirements for 
planning and evaluation support the notion that planning and evaluation in vocational 
education are driven by legislative mandate. Thus, planning and evaluation continue to be 
compliance-oriented activities at both the state and local levels. 

/ 

Summary 

This section has presented operational definitions of planning and evaluation and a 
general overview of the statutory basis of planning and evaluation in vocational education. 
There is a legislative history of increases in the amount and specificity of required planning 
and evaluation. With these increased responsibilities, states and localities have attempted to 
incorporate multiple perspectives regarding the mission of federally supported vocational 
education. However, the separation of legislative language dealing with planning 
requirements from that dealing with evaluation, the dispersion of responsibility among 
multiple agencies and levels, and the lack of clarity of a dominant mission for vocational 
education all appear to have contributed to the major problem that is the focus of the present 
study: the general failure of establishing planning and evaluation as mutually supportive 
activities. 
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PLANNING 



There has been no lack of desire among vocational educators to make the planning, 
delivery, and evaluation of vocational programs more rational. As reviewed in the first 
section of this document, planning in vocational education has a statutory history dating to 
the original Smith-Hughes Act of 1917. Revisions of that and subsequent acts have 
increased emphasis on planning (and evaluation) with additional requirements and with the 
dispersion of responsibility and oversight among agencies. 

This section presents an overview of planning in vocational education in four areas: 
(1) definition of planning, (2) questions faced by vocational education planners, (3) 
planning theories or approaches, and (4) analysis of planning approaches for application to 
vocational education. 

Definitions of Planning 

A plan may be defined simply as a map from which to proceed from a present state 
to certain future states. As geographical maps come in different forms — depending on the 
intended use and the terrain — so does planning. Within organizations, the meaning of 
planning is shaped in the actual formation of strategies. Given this general orientation, the 
meaning of planning varies. For example, 

1 . planning may be conceived simply as future thinking — that is, as taking the future 
into consideration; 

2. planning may be equated with following formal procedures such as a program 
planning budgeting system (PPBS) and articulating results; 

3. planning may also be understood in terms of programming, where the task is to 
justify and elaborate an accepted strategy rather than to conceive one. 

Mintzberg (1983) described planning as organizational foresight — "drawing 
strategists together to speculate about the future and making sure that such speculation takes 
place" (p. 323). Beebe (1966) defined planning as the exercise of foresight. Hudson and 
Davis (1976) included the development and statement of goals; determination of policy and 
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program alternatives; assessment of costs and resources and evaluation of outcomes or 
effects; and the monitoring of allocations, decisions, and implementation activities. 

As stated in the first section of this review, planning is considered to involve — 
ideally at least — three major functions: (1) the setting and modification of goals and 
purpose; (2) the systematic determination of policy and program alternatives; and (3) the 
use of appropriate needs, cost, resource, output, and outcome data in monitoring programs 
and providing feedback for the planning process on a continuing basis. It becomes 
apparent that as one's definition of planning becomes more comprehensive, the differences 
between planning and evaluation become less clear. 

Planning in vocational education has been criticized for being viewed primarily as a 
means for obtaining resources, a view which results in a "compliance" mentality among 
vocational planners (Strickland & Asche, 1987). It has also been observed that planning at 
lower levels is modeled on or dictated by the requirements of higher levels (Copa & Moss, 
1983). A third common observation has been that vocational education planning has 
attempted to implement a rational-comprehensive approach to the almost total exclusion of 
other approaches. To the extent that these characterizations are accurate, they represent a 
rational response to federal legislation and regulations in which states and localities are 
viewed as sharing certain similarities and in which funding is contingent upon compliance. 
A reasonable hypothesis would be that vocational education has not been well served by 
such a restricted view of planning, especially considering the large portion of local and state 
planning resources that have been devoted to rationalizing programs that are only minimally 
funded by the federal sources. 

Most observers agree that the primary thrust of planning in vocational education has 
been to make the process more rational. Several assumptions underlie the notion of rational 
planning and have been summarized by Schmidtlein (1983): 

1 • The technical analysis of problems, goals, and change strategies 
produces sufficient understanding and agreement to permit the 
establishment of goals and priorities. This implies that conflicts are 
based principally on lack of understanding rather than on 
fundamental disagreements over values and self-interests. 

2. The area subject to planning is sufficiently understandable so that 
crucial relationships can be determined, technologies for change can 
be developed and outputs can be identified and measured. 

3. The economic, social, human, and information resources necessary 
to design, implement, and evaluate plans must be available. If the 
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money, the ideological commitment, the human talent, and the 
information technology are inadequate, planning will, to that extent, 
be inadequate. 

4. The rate of change, deadlines, and competing priorities in the 
environment must allow sufficient time for analysis, an essential 
element in planning. 

5. The consequences of planning must serve the requirements of the 
roles of key participants, (pp. 52-53) 

Each of these assumptions has relevance for vocational education. First, purely rational 
approaches to planning will not resolve fundamental disagreements among decisionmakers 
that are based on values or self-interests. Second, rational planning assumes a certain state- 
of-the-art knowledge in an area. If research has not been conducted or has not been fruitful 
in identifying critical variables and relationships or if the variables cannot be reliably 
measured, rationalization of the planning process may be seriously restrained. In 
vocational education, the quality of available data on programs, students, and the labor 
market has been questioned. Third, planning requires human and fiscal resources and 
commitment on the part of decisionmakers on the basis of planning outcomes. Finally, 
planning must operate in a climate that is realistic, both in terms of the demands placed on 
planners and in terms of the compatibility of the planning process with the decision-making 
structure within an organization. 

Concerns Faced by Vocational Planners 

Several very basic concerns have faced vocational education planners over the 
years. Perhaps the most basic is alluded to above: the degree of belief in the viability of 
rationality as it applies to future events. Mannheim (1949) distinguished between 
functional and substantial rationality. Functional rationality relates ends to means; whereas, 
substantial rationality attempts to deal with the appropriateness of ends themselves by 
studying the structural relations of the social system. Defining the role of planning in 
vocational education is still an attempt to deal with this very basic issue of whether the 
primary function of planning is allocative (serves to increase the efficiency of means-ends 
linkages) or substantive (questions ends themselves even though they are often legislatively 
determined). 

Another concern is the viability of bureaucratically centralized planning in a social 
structure (i.e., education) characterized by lack of central control. Related to this is the 
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question of whether planning should be largely inductive (from local to state to federal) or 
deductive (state and local planning follows an overall federal design for vocational 
education) (Moss, 1983, p. 14). Yet another issue relates to the utilization of planning 
since planners may serve to clarify options or may use coercive power to reduce individual 
options for the (supposed) collective good 

For instance, the following were posed as "front-end" questions faced by 
vocational educators (Aschc, 1985): 

1. Should the planning process extend beyond that necessary to 
comply with federal and state requirements? 

2. To what extent should planning deal with the appropriate ends of 
vocational education? Should planners be restricted to maximizing 
allocative decisions? 

3. To what extent should planning be centralized? Which decisions 
should be made in a top-down fashion; which in a bottom-up 
participatory or advocacy fashion? Which should be made within 
program areas and which across-the-board? Is a mixed model 
workable for vocational education planning? 

4. To what extent can empirical decision theory methods be used cost- 
effectively? How much of the process should be data based; how 
much should be social process? (p. 12) 

In order to deal with questions such as these, it is helpful to have a broader view of 
planning approaches or models than has traditionally been the case in vocational education. 
Major theoretical developments in planning reflect different philosophical assumptions and 
parallel major historical periods. Vocational education planning, however, has tended to be 
closely tied to the requirements of major federal vocational education acts and has reflected 
neither the complexity nor the diversity of planning theory in general. The following 
provides a brief synopsis of several major traditions in planning theory. 



Planning Theories or Approaches 

Friedmann and Hudson (1974) provided a classification of major approaches to 
planning that highlights major differences among them. Some approaches grew out of 
"traditional rationalism" and focused on how decisions can be more rationally made. In 
this sense, most decision making was directed toward allocating resources and seeking 
optimization. In contrast, some approaches were based in organization development and 
focused on ways to achieve change in organizational structure and behavior. Rational 
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decision theory was characterized as improving the quality of decisions with little regard for 
implementation. Organization development focused on the developmental/experimental 
process of interpersonal interactions and on organizational structure rather than on the 
decisions themselves. 

Friedmann and Hudson's third classification, "tradition of empiricism/ 1 focused on 
the actual behavior of large scale political and economic systems. An oversimplified 
characterization of these approaches is that empirical investigation of operating systems will 
reveal the impact of variables that can then be used in predictive models. Rational decision 
making remains the core of this approach with such decisions being based on empirical data 
gathered from operating systems. Microeconomic simulation models as originally 
proposed by Orcutt (1961) and more recently developed for vocational education planning 
by Beaulieu and Erickson (1977) and Morgan and Cohn (1977) are examples of rational- 
empirical model applications in this category. 

The history of planning theory reveals a sensitivity to the major issues of a given 
period: heavy dependency on rationality, logic, and decision theory up to the 1960s with a 
shift to greater incorporation of participation; advocacy and decentralization in the latter 
1960s and early 1970s (Davidoff, 1966; Friedmann, 1973; Simmons, 1974). Vocational 
education has traditionally adopted the rational-empirical view of planning with a high 
degree of centralization and with the primary purpose of allocating resources to competing 
interests. Beyond state plan preparation and accountability reporting, however, what actual 
planning does take place is often informal, incremental, and decentralized (Asche, 1985). 

Until recent years, most planning literature has focused on "rational" models and 
reveals four to six major subdivisions of activity. Koontz (1963) listed the following five 
steps in the rational planning process: 

1 . the identification or specification of the goals or objectives for 
planning, the ends to be sought; 

2. the determination of planning premises, specification of constraints, 
and forecasting of relevant exogenous variables; 

3. the identification of alternative courses of action which might be 
undertaken; 
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4. the evaluation of alternative courses of action through forecasts of 
consequences and the assignment of relative values to various 
consequences; and 

5. the selection of a course of action through the application of some 
decision-making rule. 

O'Reilly (1975) identified several common steps in the vocational education 
planning process. First, states collected data about the previous year. Second, the data 
about the previous year's activities was evaluated in terms of previously established goals 
and objectives. Third, based upon an estimate of anticipated income, goals were 
determined for the following year. Fourth, the plan was written. Fifth, a review of the 
plan was conducted, involving persons and agencies specified by current statutes. These 
steps are similar to those proposed by Koontz (1963) for the rational approach but with less 
focus on the conscious identification of objectives for planning and without explicit 
identification of underlying assumptions and premises. 

The more recent views of planning do not present such a clear-cut blueprint. 
Hudson (1983) listed five approaches to planning: synoptic (rational-comprehensive), 
incremental, advocacy, radical, and transactive. The latter four approaches have developed 
since about 1960. None of these approaches seems appropriate for a complete planning 
system for vocational education, yet each has potential merit for addressing some of the 
recurring criticisms of planning practices in vocational education. Based on Hudson's 
(1983) account, a brief review of each approach is presented below. 

Synoptic Planning 

Hudson stated that the synoptic or rational-comprehensive approach to planning is a 
dominant tradition and that most other approaches represent modifications or reactions to 
this approach. He listed four classical elements: (1) goal setting, (2) identification of 
policy alternatives, (3) evaluation of means against ends, and (4) implementation of 
decisions. Techniques of planning such as regression, econometric modeling, Markov 
chains, and simulation may be nested within one or more of the basic elements. Most 
recent formal efforts in vocational planning appear to model the basic elements of the 
synoptic approach. The primary focus, however, has been on "empiricising" the middle of 
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the model — that is, on using program, student, and labor supply and demand data in a 
forecasting model to improve predictions of needs for expansion or contraction of existing 
programs and processes. Few recent efforts have seriously incorporated formal planning 
procedures in the goal setting stage of the model and consideration of policy alternatives 
has operated informally and within pre-established historical and legislative bounds. 



Incremental Planning 

Lindblom (1959) has been a chief proponent of the incremental approach to 
planning, labeling it the "science of muddling through." He stated that policy decisions are 
better understood and better accomplished using this approach, which uses the "push and 
tug" of established institutions in the normal bargaining processes. Incrementalists have 
claimed that synoptic planning is insensitive to existing institutional performance 
capabilities, is reductionist, fails to appreciate the cognitive limits of decisionmakers who 
really work by successive approximation, and fails to recognize pluralistic public interests. 
According to incrementalists, synoptic planning is biased toward central control. 
Braybrooke and Lindblom (1983) outlined eight characteristics of incremental planning: 

1 . Choices are made in a given political system, at the margin of the 
status quo. 

2. A restricted variety of policy alternatives is considered, and these 
alternatives differ only incrementally from existing policy. 

3. A restricted number of consequences are considered for any given 
policy; at any one point the analysis of consequences is quite 
incomplete. 

4 . Adjustments are made in the policy objectives in order to conform to 
given means of policy, implying that ends and means are chosen 
simultaneously. 

5. Problems are reconstructed, or transformed, in the course of 
exploring relevant data. 

6. Analysis and evaluation occur sequentially with the result that policy 
consists of a long chain of amended choices. 

7. Analysis and decision making are remedial; they move away from 
negatively perceived situations and toward known objectives. 

8. Analysis and decision making are socially fragmented; they take 
place at a very large number of separate points simultaneously, (p. 
54) 



The incremental approach is probably a fair approximation of the way in which 
much of the planning in vocational education actually takes place. This approach requires 
little central control, data collection, and decision making. It recognizes the political nature 



of the process and also recognizes that goals, values, and expectations are often the source 
of conflict. It is remedial in that planning is conducted to "fix problems" or, as stated 
above, to move away from negatively perceived situations. It allows for bargaining by 
affected parties and is probably best characterized by Lindblom's "muddling through" 
description. 

Etzioni (1967) proposed an approach called "mixed scanning" which he 
characterized as neither as Utopian as the rationalistic models nor as conservative as 
Lindblom's incremental model. Etzioni presented the idea that in a society where 
consensus is low, rationally determined policies of central authorities can only be carried 
out through the exercise of excessive coercive power. The outcome of such an 
"overmanaged" system is social alienation and instability. Etzioni's criticism of rationalistic 
models parallels the problems cited by Friedmann and Hudson (1979) for rational decision 
theory — namely, the problem of adequate knowledge of variables and relationships; the 
debate as to whether a community welfare function actually exists; and the problem of the 
assumption that just because a "scientific" or "rational" decision is made, it will be 
implemented. 

Etzioni (1967) cited two operational problems with the incremental approach. First, 
decisions by consent among partisans without a society-wide regulatory agency will reflect 
the interest of some groups to the exclusion of others since power is not evenly distributed. 
Second, incrementalism focuses on the short term and neglects innovations. Both 
criticisms reinforce the observation that incrementalism reinforces and maintains the 
existing condition. Finally, Etzioni criticized the incremental approach for its failure to 
recognize the importance of large fundamental decisions that fall outside the incremental 
decision model. This concern parallels the distinction made by Mannheim (1949) between 
functional and substantial rationality and by Friedmann's (1973) allocative and innovative 
planning. 

Etzioni used this dichotomy of decision types and levels as the basis for his "mixed 
scanning" approach. In this approach, two or more levels of "scanning" are used to deal 
with planning problems which inherently vary in level, complexity, and import. The 
broader scan lacks detail but has the power to detect areas requiring attention. These areas 
may be addressed through an incremental approach but with the advantage of the broader 
view of the whole system. The mixed scanning proposal thus appears to be an attempt to 
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deal simultaneously with the long- and short-term nature of problems, incremental as^Well 
as fundamental decisions, the need for centralized consensus building without oven 
management, and the incorporation of both allocative and innovative planning. 



Advocacy Planning 

Advocacy planning came about primarily as a reaction to approaches which did not 
address the unequal distribution of power in society. Theoretically, advocacy planning 
should produce multiple plans that address the pluralistic needs of groups within the larger 
society. Within vocational education, the needs of clients from targeted groups or special 
populations may argue for an advocacy planning approach. 

Peattie (1968), however, illustrated some of the problems in implementing this 
theory. She cited the frequent lack of homogeneity in the needs and interests of supposed 
subgroups and the potential common interests of diverse groups as examples. In a graphic 
illustration of her point, she cited the participation of members of the John Birch Society 
and of Students for a Democratic Society in the same picket lines protesting urban renewal 
policies in Boston. Advocacy planning, since it is based on an adversarial model, may also 
be more effective in blocking inequitable policy than in promoting equitable or 
compensatory policy. 



Radical Planning 

As in the other proposals for planning reform or modification, advocates of 
"radical" planning have started from a base of criticism of the rational-comprehensive 
model. The most common criticisms were that the rational model (1) is elitist in that it sets 
the planner apart from the world he or she is to plan, (2) is centralizing in that the type of 
control necessary requires centralized activity, and (3) is change-resistant in that the attempt 
is to eliminate all but preprogrammed social change (Grabow & Heskin, 1973, p. 108). 

Radical planners advocated that "society be reorganized so that the maximum 
number of decisions possible can be within the reach of as many people as possible" 
(Grabow & Heskin, 1973, p. 109). The notion of "paradigm shift" as proposed by Kuhn 
(1962) is viewed as positive and is in the interest of human development and thus should 
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be facilitated rather than resisted by planners. A third area that is seen as compatible with 
decentralization and changing world views is advocacy of innovation and experimentation 
in social organization. This last area is often perceived as threatening in that radical 
planners do not feel constrained to work within existing structure and organizational 
patterns. The radical view of planning stresses cooperation and communal decision- 
making rather than competition. It accuses comprehensive planning of placing too much 
emphasis on competition and economic growth objectives rather than embracing an 
ecological ethic or quality of life focus. Planning is seen as more an act of facilitating than 
predicting, so the planner is viewed as an active change agent rather than an isolated 
scientist or technician. Primary benefits derive from spontaneity, innovation, and learning 
from the resulting changes. Progress would occur as a result of learning rather than from 
prediction and central control. 



Transactive Planning 

Transactive planning grew out of the turmoil of the 1960s and was in some ways 
designed to ameliorate problems of social alienation addressed by the mixed-scanning and 
radical planning approaches. Friedmann (1973) presented an extensive analysis of the 
conditions in postindustrial society that called for greater communication between planners 
(technical experts) and clients through a process he called "transactive planning." His 
rationale was based in two areas of crises: values and knowledge. 

The movement from a history of scarcity to an. era of abundance has been 
accompanied by a diffusion or loss of clarity and simplicity of values. The growth of 
cultural pluralism has made planning more complex in that the determination of a common 
universal good is often not possible. Earlier systems of planning advocated separation of 
values from the technology of planning; transactive planning calls for reintegration. Social 
alienation has led to activism which often blocks implementation of centrally developed 
plans. Friedmann (1973) cited examples of the blocking of urban renewal plans for several 
cities as examples of a weakening link between central plans and implementation. 

A crisis of "knowing" has resulted from the growing scale and complexity of 
sociotechnical systems, the accelerated rate of change in social systems and technology, 
and the growing gap between experienced and observed reality. Individuals and groups 
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must now depend more upon "processed" (symbolic) knowledge and less on experiential 
knowledge. However, the growing gap between the technical expert and the client 
exacerbates the problem of communication. 

Friedmann (1973, p. Ill) proposed that the solution lies in forging a personal 
relationship between expert and client through the process of transactive planning. In this 
interchange, the values of the people affect the planners and planning process, but the 
expertise of the planners informs the client — a learning process that helps link processed 
knowledge to action. 

To illustrate, one might visualize a dichotomy. At one extreme of this dichotomy, 
actions are determined through political processes without benefit of scientific technical 
knowledge. At the other extreme, complete dependence on technical knowledge without 
the value system of a pluralistic society leads to the exercise of coercive power and an 
"overmanaged" society (Etzioni, 1967). Transactive planning is proposed to connect the 
political process with the technology of planning to achieve "social guidance." It should 
connect those who primarily have processed knowledge with those who operate on 
experiential knowledge. Friedmann (1973) contends that the transactive style of planning 
can be applied to both allocative planning (the distribution of limited resources among 
competing users) and innovative planning (actions that produce structural changes in the 
guidance system). Transactive planning, at least in theory, offers the potential of 
addressing the apparent gap in vocational education between federal and state planners and 
between public school personnel and clients of the educational system. 



Allocative Versus Innovative Planning 

The guidance of the vocational enterprise might be likened to Mannheim's "social 
guidance" as described by Friedmann (1973) in that both functional rationality (relating 
means efficiently to given ends) and substantial rationality (examination of the ends 
themselves through intelligent insight into the workings of the social system) are required. 
Often, planning has been defined to include only functional rationality. 

A parallel exists in the evaluation literature. Some feel that the only appropriate role 
of the evaluator is to supply information to decisionmakers. Planners who feel the 
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planning mission is one of functional rationality can avoid value judgments by simply 
accepting goals as established through the political process. The work of the planner is 
thus limited to the allocation role — examining implications of alternative allocative schemes 
for attaining the given goals. A further restriction is not uncommon. Resources are set by 
the political process, leaving the planners with the purely functional role of "managing" the 
allocation to get the best (or at least an acceptable) fit between limited resources and given 
goals (e.g., efficiency of allocation). Friedmann (1973) labeled such thinking "an exercise 
in self-delusion" (p. 58). His "allocative" and "innovative" planning functions seem to 
parallel Mannheim's functional and substantial rationality. The following discussion 
presents a rationale for the notion that guidance, not just efficiency, of the enterprise is a 
legitimate and, in fact, necessary function of planning. In other words, the vocational 
education planning function at this point in time must be expanded to include a greater 
focus on innovative (substantive) planning. 

The sociology of knowledge indicates that the "world view" of any group within a 
society tends to restrict the ability of that group to functional rationally, thereby preventing 
long-range planning of a visionary or Utopian nature. Mannheim's notion that a group of 
intelligentsia might lack the encumbrances of a group membership may be more a matter of 
convenience than of reality. Today's "scholar," operating in the politicized university or 
state department of education, may lack the intellectual autonomy desired by Mannheim. 
The challenge to designers of planning systems or models thus becomes one of somehow 
insuring that innovation is not extinguished in the press for efficiency or political 
expediency. 

Friedmann (1973, p. 17) defined allocative planning as the distribution of limited 
resources among competing users. State level management of vocational education has 
evolved toward a restricted form of allocative planning by trying to satisfy the educational 
establishment throughout the state while also demonstrating compliance with federal and 
state mandates and constraints. The state establishment consists of various constituencies 
(parents, vocational teachers and administrators, students, professional associations and 
unions, business and industry, and general education administrators), each group with its 
own world view and set of vested interests, operating in a rigid organizational structure. 

Various models for planning vocational education within the existing constituent 
and structural confines have been developed, used, and generally abandoned. In some 
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cases, valiant original efforts have been modified to provide required state plan 
"compliance" documents, thereby sacrificing their original comprehensive planning 
mission. These models have exclusively drawn from the rational-synoptic or incremental 
traditions and have served the restricted role of allocation. They can and have worked 
insofar as the planning role has been defined as optimizing — or at least "satisfying" 
(Simon, 1945) — allocation decisions in a relatively stable environment. They have been 
inadequate in times of rapid change (new legislation, technological and economic change, 
educational reform). Even under optimum conditions of stability and funding, the cost 
effectiveness of increasing conceptual or mathematical sophistication is debatable. 

The "fatal flaw" of extant models, however, is not one of efficient allocation. It is 
the inability to encourage or even tolerate what Mannheim (1949) called substantial 
rationality or Friedmann's (1973) innovative planning. Present models and practices accept 
existing structure and constituency. Because they involve a priori historical or physical 
restrictions (often termed "reality" or "feasibility"), these models tend to perpetuate the 
existing condition or only move incrementally away from the negative (and seldom toward 
the positive). The mission is problem solving at the exclusion of opportunity seeking and 
is oriented toward defense and maintenance instead of offense and change. 



Analysis of Planning Approaches for Applicability to Vocational Education 

The assessment of the potential of each of the theories and approaches to vocational 
education planning would be simple if vocational education were a narrowly defined, 
independent entity. It is, however, a changing and complex network of people and 
services. It falls under several levels of control and funding and is related to the 
community and other school and social programs in ways that are highly sensitive to local 
needs and characteristics. Vocational education does not control the labor market nor does 
it presently control the pretraining aptitudes or competence of its students. Insofar as 
federal support is concerned, vocational educators have minimal input in the legislative 
process that sets its national missions, selects its clientele, and determines the criteria on 
which it will be evaluated. In addition, vocational legislation has tended to encompass two 
sometimes conflicting perspectives of the mission of vocational education, an employment- 
based perspective and a school-based perspective (Thompson, 1973). 
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An adaptation of an old planning paradox seems appropriate: Where change is 
rapid and knowledge is minimal, the need for planning is at a maximum but its 
effectiveness is at a minimum. Where conditions are stable and information is readily 
available and reliable, planning is maximally effective but not needed! The relative value 
and feasibility of planning must be viewed for what it is expected to accomplish and thus 
for how it is defined If planning is defined only in terms of its ability to predict the future, 
its ability to exercise control or to move a system toward a preconceived desirable 
configuration — the paradox seems to hold. Planning's potential to maintain the status quo 
in the face of rapid change or to protect existing power structures is suspect Much of the 
present loss of confidence in the value of planning may be attributed to its inability to do the 
impossible — isolate a subsystem such as vocational education from the larger educational 
and societal systems of which it is a part. 

A critique of planning approaches must therefore not be based on an appraisal of 
their value in maintaining, protecting, or even expanding vocational education. It should 
not start with the assumption that the future is one in which vocational education stays the 
same while everything about it changes. 

Hudson (1983, pp. 29-34) provided an analysis that used six basic criteria in 
assessing the adequacy of several basic planning theories. The criteria included (1) the 
extent to which the public interest is incorporated, (2) the attention given to the human 
dimension of policy impact, (3) the feasibility of using the particular approach, (4) the 
action-generating potential, (5) the extent to which a particular theory addresses substantive 
social theory, and (6) the extent to which theory is self-reflective. This review used 
Hudson's first five criteria in a somewhat modified form and has added several others 
which the review reveals may be critical to vocational education. The revised set of criteria 
are defined as follows: 

1. public interest — extent to which an approach incorporates the 
public/client interest 

2. human impact — extent to which an approach takes into account the 
probable human impact of actions or policies 

3. feasibility of application — the relative probability of being able to 
use a given approach in the present context 
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4. implementation — extent to which an approach facilitates 
implementation of plans or policies developed 

5. substantive theory — extent to which an approach incorporates 
substantive theories of social and organizational change 

6. mission and goal setting/clarification — extent to which an approach 
assists in definition or redefinition of vocational education missions 
or goals 

7. leadership involvement and development — extent to which an 
approach facilitates the utilization of expertise and development of 
new leaders 

8. local and regional variability — extent to which an approach can 
accommodate and encourage local and regional variability 

9. experimentation and innovation — extent to which an approach 
facilitates experimentation and innovation 

10. generation and use of objective data — extent to which an approach 
generates and uses objective data in the decision process 

Figure 1 is a summary assessment of the relative strength of each of the five basic 
planning approaches for each of two basic planning functions in vocational education. 
Although based on Hudson's (1983, p. 31) original concept, the ratings have been revised 
to reflect subjective judgments based on application of the approaches specifically to 
vocational education planning. Hudson's original conceptualization has also been modified 
by addition of a third dimension (innovative and allocative planning) and by criteria six 
through ten. It is doubtful that a high degree of agreement on the judgments reflected in the 
table could be achieved among vocational education planners. Its primary purpose is to 
assist planners in their own assessment of the relative strengths and weaknesses of the 
various approaches under the varying conditions of the states. 
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Summary 



Vocational education's "world view" of planning has been historically restricted so 
that it might be characterized as rational-empirical, centralized, and allocative. It has been 
pointed out that such an approach is maximally effective with political and economic 
stability, accurate data, and mission consensus. It was also observed that such a view 
restricts the group to functional rationality, thereby preventing long-range planning of a 
visionary or Utopian nature* 

The literature regarding planning theory provides several approaches to planning 
that attempt to address the limitations of the "rational-comprehensive" approach under 
conditions more descriptive of the "real world" in which vocational education operates. 
The literature on vocational education planning, however, does not indicate any wide-scale 
experimentation or adoption of such approaches. 

If the linking of planning and evaluation were viewed largely as an exercise in 
joining two separate functions or agencies within the bureaucratic organizational structure, 
the present modal approach to planning might not be an encumbrance. If, however, linking 
is viewed as a conceptual as well as structural joining, resulting in mutually supportive 
activities rather than distinct functions, aspects of the incremental, advocacy, radical, and 
particularly the transactive approaches seem relevant. This view may be held by those 
involved in the informal planning and evaluation activities that take place, but the view is 
not treated explicitly in the formal vocational education literature. 



EVALUATION 

Evaluation or, more specifically, program evaluation can be understood simply as 
the use of social science methods to monitor and assess the operation of public programs. 
The basic function of program evaluation is to determine whether a specific program is 
achieving its objectives and, if it is not achieving its objectives, to identify the reasons for 
failure. However, program evaluation should not be constricted by this conceptualization 
because evaluation may begin before a program or policy is implemented or because it may 
touch on issues that were not envisioned at the time the goals of the program were 
formulated. As Chelimsky (1983) observed, program evaluation seeks to improve the 
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basis for policy making rather than to redirect policy. More often than not, program 
evaluations supply the proof of legitimacy, effectiveness, and efficiency necessary to 
justify continued program support. 

The literature on program evaluation can be broken down into four major 
discussions: (1) the scope and nature of evaluation (goal directed or summative versus goal 
rationalizing or formative), (2) the method of evaluation (social scientific versus 
naturalistic), (3) the politics of evaluation (status quo versus social critique), and (4) the 
utilization of evaluation (formal versus unobtrusive). 

The Scope of Evaluation 

In an attempt to map the field of evaluation, Palumbo and Nachmias (1983) asked 
whether there is an ideal paradigm. Their answer was that there is no ideal paradigm and 
that "the dominant model is both methodologically and institutionally inadequate" (p. 78). 
They attributed this inadequacy to the unrealistic assumptions of the dominant synoptic 
model. For example, the goal-directed paradigm depends on the presence of an ideal 
program manager committed to achieving program goals. Yet, a host of circumstances 
serve to undermine this idealized role. Making use of evaluation results that either are 
inconclusive or disagree with immediate political objectives and ideological dispositions 
frustrates and delimits even the most committed decisionmaker. 

From a historical point of view, the evolution of evaluation research spans a variety 
of sponsors, purposes, and methods that characterizes each evaluation paradigm. Daniels 
and Wirth (1983) characterized this evolution in terms of what they refer to as the locus, 
focus, and modus of each evaluation paradigm. The first paradigm, "evaluation research as 
efficiency and the New Deal," was identified as being prevalent between 1910 and the 
World War n. The locus of this paradigm was industrial research, inspired primarily by 
Frederic Taylor and his time-motion studies and by governmental agencies responsible for 
carrying out New Deal programs. The focus was also twofold: efficiency in productivity 
and evaluation of social change. The modus for these research efforts hinged on early 
applications of experimental, applied social research. 
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After World War II and up to about 1963, the prevailing paradigm was "evaluation 
research as field research," As the title suggests, field research in sociology and 
psychology and even in the armed services predominated. Social-psychological and 
behavioral-attitudinal change studies became the focus of evaluation research and social 
programs. Experimental designs left the laboratory and were introduced in field settings 
primarily through the work of Kurt Lewin and his use of both qualitative and quantitative 
methods. 

The third paradigm, "evaluation as social experimentation," emerged with the 
introduction of Lyndon Johnson's Great Society and social action programs- With its basis 
in social action programs, the focus of evaluation was on public policy, the "turbulent 
setting" of social programs, and the problem of evaluation utilization within the policy 
arena. The emphasis was social experimentation, spurred by the classic works of 
Campbell and Stanley (1963). 

Finally, the fourth and current paradigm, "evaluation research as a policymaking 
component," identified by Daniel and Wirth emanated from its predecessor. The locus of 
evaluation moved to all levels of federal and state governmental activity. The focus found 
its form in the congressional power of oversight. Subsequently, federal legislation 
translated this oversight into a centralized, rationalized approach to policymaking. The 
modus of this paradigm is characterized by a myriad of methods and techniques, not to 
mention a fine-tuning of experimental and quasi-experimental techniques. 

Given this historical perspective, the evaluation strategy is considered to vary 
according to time and organizational context. For example, if the focus is on past 
decisions, then impact or output evaluation is followed. This is often designated as 
summative evaluation. If, on the other hand, the focus is on present decision making, then 
process or system evaluation and comparative case analysis are followed. This is often 
designated as formative evaluation. Formative evaluations provide the information 
necessary to design and/or modify service delivery systems while they are in operation. 
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Methods of Evaluation 

The interaction between paradigms and strategies gives rise to different types of 
evaluation. Perkins (1977) identified six basic types of evaluations: 

1 . Strategic evaluations are concerned with underlying causes of social 
problems, focusing on "implicit theories" as a basis for broad 
ameliorative programs. 

2. Interaction effect assessments attempt to establish the relationship 
between program intervention and outcomes or, in some cases, the 
processes involved in producing those outcomes. 

3. Compliance evaluations examine the consistency of program 
objectives with broader legislative aims and attempt to ensure that 
public funds are allocated according to policy guidelines. 

4. Program design evaluations test the measurability of program 
assumptions, the overall logic of the program approach, and the 
assignment of responsibility and accountability for program results. 

5. Management evaluations focus on the efficiency and effectiveness 
with which managers deploy available resources to achieve program 
objectives. 

6. Program impact evaluations deal with the program delivery system 
and the relation between program results and the legislated goals and 
program objectives. 



The last type, program impact evaluation, is the dominant mode of evaluation in 
practice. Morell (1979) elaborated further on this type, breaking it down into three 
categories: client-type comparison evaluation, follow-up evaluations, and modality test 
evaluation. Client-type comparison evaluations are concerned with the relative effect of a 
program on various subpopulations or between the characteristics of those who receive 
treatment versus those who do not. The focus of interest is on sociopsychological factors 
that differentiate one group of people from another. Follow-up evaluations are directed at 
people who have left the immediate focus of "treatment" that they have received. The 
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purpose is to see how people fare once the immediate effects of the program or treatment 
have been removed. Modality test evaluations are used to estimate the relative effects of 
some method (or methods) which cause an identifiable, desired change. 

Finally, the conduct of research in evaluation as well as the validity of its findings 
also reflect contending approaches. The central issue is the debate over rigor versus 
relevance, more popularly phrased as quantitative versus qualitative research. The rigor 
side of the argument asserts strict adherence to the methods of falsification testing, 
verification, and replicability for a particular evaluation study to be scientific. The 
relevance side, on the other hand, stresses the importance of intersubjectivity, participant 
observation, and triangulation for evaluation research to be naturalistic* 

Recognizing this variety in the evaluation enterprise helps one to appreciate the 
strategic use of evaluation. Thus, rather than being wedded to a single view of evaluation 
(i.e., summative or formative) one can begin to raise questions about the other potentials of 
evaluation efforts (i.e., linkages with planning). This can be accomplished by conceiving 
of evaluation as a built-in feedback mechanism in the planning process. Obviously this is 
more easily said than done. The doing requires recognition of the political nature of 
evaluation and, in particular, of the problematic nature of evaluation utilization in program 
decision making. 

Politics and Evaluation 

A rich literature has emerged surrounding the debate over the nature of politics in 
evaluation. The primary contention revolves around the question of whether evaluation 
itself is a political act or a tool of politics. Those who hold that evaluation is a tool of 
politics argue that evaluations become part of the political decision-making process 
surrounding the program being evaluated (Palumbo, 1987, p. 12). Weiss and Bucuvalas 
(1977, pp. 525-533) observed that, in evaluation, political considerations intrude in three 
major ways. The first intrusion concerns the origins of evaluation — where the specification 
of evaluation purpose and evaluation criterion emerges from the decisions reached through 
the political process. The second concerns the role of evaluations in decision making — 
where evaluation either mobilizes program support or uncovers the need for program 
modification. The third way concerns the ideological biases built into the parameters of 
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evaluation — where the expertise of the evaluator confers focus on some variables while 
ignoring others. The message being transmitted is that those which are ignored are 
relatively unimportant 

Those who hold evaluation as a political act focus their attention on the role of the 
evaluator. Barry and Rae (197S) observed that "evaluating is assigning value to things 
roughly speaking, determining whether they are good or bad. We shall take political 
evaluation to consist in the first instance in assigning value to alternative policies, laws, or 
general decisions binding on a collectivity" (p. 340). Assigning values to alternative means 
also involves a judgment about the goals themselves. Mettsner (1972, p. 859) argued that 
evaluators must focus on and explicitly incorporate political dimensions in their studies in 
older to carve out a role in the policy decision process. Playing this political role is neither 
simple nor straightforward. As Palumbo (1987) noted, 'The political dilemma facing 
evaluators is to steer a course between recognizing the political reality of evaluation and 
retaining the symbolism of neutrality" (p. 20). Patton (1987) has been more emphatic 
about the role evaluators should play. He wrote, "The issue for evaluators is not whether 
evaluations are political, since they cannot be otherwise. The issue is how explicitly the 
evaluator deals with, facilitates and negotiates political considerations in the context of 
concerns for use, practicalities, rigor, integrity, accuracy, fairness and credibility" (p. 143). 



Evaluation Utilization 

While the literature diverges regarding the political nature of evaluation, there is one 
issue over which there is agreement This has to do with the fact that evaluation utilization 
is problematic. Nachmias (1989) noted that "the criterion for assessing utilization is built 
into the definition of evaluation: Evaluation research is meant for immediate direct use in 
improving policy decision" (p. 1165). He further noted that "the very few published 
systematic empirical studies of evaluation utilization tend to support the proposition that 
utilization — in the sense of influencing program decision making and management-— docs 
occur, but not in the manner proposed in the early evaluation literature" (p. 1 166). Such 
experiences tend to lend a rather amorphous or symbolic perspective to evaluation rather 
than to support the view of evaluation as a substantive or direct endeavor (Weiss, 1988). 
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Various classifications for describing the variety of uses that evaluation research 
serve have been forwarded* Three broad categories of factors that influence utilization can 
be identified: (1) human factors (attitude, commitment, expertise, organizational position), 
(2) context factors (task environment, fiscal climate), and (3) evaluation factors (evaluation 
procedure, quality of information)* Most classifications, however, focus on the types of 
utilization rather than the influences on utilization. 

Smith (1988) posed a synthesis called "the many faces of utilization," which 
classified utilization of evaluation according to four dimensions: (1) directly observable 
versus perceptual, (2) process versus results, (3) immediate versus long term, and (4) 
partial/incremental versus holistic. Morell (1979) extended the variety of uses further by 
identifying four different uses — realistic expectations, improved service, long-range 
planning, and political utility — by mode of program evaluation: 

Although no categorizations will be sharp and clear, four basic uses can be 
discerned: as a tool for setting realistic expectations for a program or 
treatment; as a method of helping organizations to improve the quality of 
service which they deliver; as a guide for long-range planning; and as a 
political tool in the sociopolitical decision-making process. The evaluation 
design concerns which an evaluator must consider will differ depending on 
which aspect of usefulness is to be maximized. Further, the type of 
evaluation which is to be conducted (client comparison, follow-up, modality 
test) will also influence strategies for increasing any given aspect of 
usefulness, (p. 90) 

A succinct, graphical view of Morell's description is presented in Table 4.1 of his book 
(Morell, 1979, p. 91). A brief statement describing the interaction between each use and 
each type of evaluation is provided in the table to assist the reader in understanding 
Morell's presentation. 

In the context of vocational education, Strickland and Asche (1987, p. 17) 
synthesized the literature into three major categories: instrumental — direct and documented 
use in decision making; conceptual — influencing a policymaker's thinking about an issue 
without documented evidence of direct use; and persuasive — using evaluation evidence to 
convince others or to defend a position from attack (Rich, 1977; Boruch & Cordray, 1980; 
Franchak, 1981; Leviton & Hughes, 1981). According to Patton, the primary weakness of 
such a synthesis is that the factors are undifferentiated in importance. Thus, no hierarchy is 
indicated that places more importance on certain factors as necessary and/or on sufficient 
conditions for evaluation utilization (Patton, 1982, p. 113). 
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Patton (1986) laid out such a hierarchy — a "three rung" hierarchy. The bottom 
rung involves improving programs by increasing the quality of decisions made. The 
middle rung consists of e valuators becoming problemsolvers, interacting with the intended 
user of the evaluation and furnishing data and information about programs that help reduce 
uncertainty in decision making (context factor). The top rung consists of the personal and 
group commitment to evaluation and the information it generates. It represents the human 
factor: leadership, interest, enthusiasm, determination, commitment, assertiveness, and 
caring of specific, individual people. 

Smith (1988) focused on the middle rung in his delineation of evaluator actions. 
He presented fifteen facilitating actions, highlights of which are listed below: 

1 . Consider utilization at every evaluation decision point 

2. Answer the questions that are asked. . . . Credibility involves . . . 
responsiveness. 

3 . Frame findings in terms of the intended users. 

4 . Focus recommendations on incremental rather than global changes. 

5. State recommendations as goals rather than delineating specific 
courses of action. 

6. State recommendations in prescriptive terms. . . . What [do] 
findings signify for future programming actions? 

7 . Make sure there is an obvious nexus between the recommendations. 

8. Avoid . . . calling into question the organization's beliefs and value 
system. 

9. Adhere to rigorous methodological standards of practice. 

10. Use a combination of approaches and methods * . . so that the 
strengths of one can mitigate the weaknesses of another. 

1 1 . [Tie] presentations of findings to the decisions to be affected. 

1 2. Make findings clear, useful, and effectively available. 

13. Rediscover the anecdote. . . . [This is] effective when used in 
conjunction with the more generalizable data. 

14. Reduce political barriers Become thoroughly familiar with the 

political process. ... Be flexible and adaptable. . . . Compromise 
is the key to achievement 

15. Couch findings within the context of other work done in the area. 
. . . [This] adds great strength to the knowledge base when findings 
are consistent across different studies, (p. 13) 

Despite Patton's (1987) advocacy perspective, the dominant view of evaluation 
contends that the political climate (context factors) places constraints on the kinds of policy 
changes that will be made, the rate at which these changes will be made, and the cost of the 
changes. In turn, these three factors define what is to be considered utilization and how 
evaluation is to be utilized. Weiss (1988), for example, pointed to the varied ways in 
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which evaluation results influence programming even when program decisionmakers do 
not implement results immediately and directly. 

In contrast to Patton (1988), Weiss (1988) was not so optimistic about the 
professional role of the evaluator. She wrote: "To ask program managers and planners to 
embrace evaluation findings fully is to ask them to bracket their years of experience and 
direct immersion in the daily world of the program, and in effect to abdicate their 
responsibility in favor of an evaluator who inevitably has only a partial view of their 
dilemmas" (p. 17). As Nachmias (1980) argued, "From the point of view of 
policymakers, evaluation is research that can help them to carry out their roles and achieve 
goals they (not the evaluators) consider important" (p. 1167)* In this process, evaluation 
information obviously competes with other policy factors. The chances of evaluation 
utilization under such circumstances increase when evaluation findings arc reinforced by 
other policy factors. Seen from this perspective, evaluation utilization can occur 
throughout the policy cycle (DeLeon, 1983). Extending the utilization concept throughout 
the policy cycle does not imply that utilization is random or accidental. Considering 
evaluation as a political enterprise conducted in a hierarchical and complex policy arena 
helps to locate the possible pattern of utilization. For example, Nachmias (1980) noted, 
"The higher the likelihood for conflict and the more fragmented the policy formation stage, 
the greater the chances for utilization to occur in the implementation stage. In such cases, 
utilization would be unobtrusive and carried out by program personnel rather than program 
formulators" (p. 1168). 

Focusing specifically on administrative control, Maynaid-Moody (1983) delineated 
a pattern of evaluation utilization. This pattern identified a dominant coalition as having 
significant influence over an evaluation process that focused heavily on accountability. The 
emphasis on goals circumvented organizational uncertainties that reinforced stress on 
assessment criteria. In short, a "closed loop" pattern of utilization was established which 
stressed accountability at the expense of organizational capacity. Evaluation in this sense 
would be enhanced through the recognition that "evaluations are not organizationally 
neutral" (p. 382). 

A final pattern of evaluation utilization is to conceive it in terms of its sensitivity to 
goal evolution (Nachmias, 1980; Daniels & Wirth, 1983; DeLeon 1983). For example, 
Cingranelli, Hofferbert, and Ziegenhagen (1980) noted that "evaluators cannot force 
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agreement regarding program goals. Neither can they stop old goals from changing or new 
ones from coming into existence, but they can recognize the likelihood that goals will 
evolve. It is the evaluators* responsibility to see that a wide range of goals are included in 
the assessment of organizational performance" (p. 1230). Similar observations pertinent to 
the implementation process in general and the educational context specifically were offered 
by Light (1980), Pressman and Wildavsky (1979), and Browne and Wildavsky (1987). 
Their research illustrated that program objectives change and evolve as they are being 
implemented 

Not an antithesis of efficiency, the goal evolution perspective's inclusion in 
utilization themes extends the measurement of goals into the realm of unanticipated or 
unintended positive and negative impacts. The importance of the goal evolution perspective 
to assessing evaluation utilization is also stressed in terms of increasing the learning 
capacity of organizations. Wildavsky's (1979) main line of argument was that 
organizations fail to learn from their past mistakes. Learning from mistakes can thus be 
considered another pattern of evaluation utilization. The importance of learning becomes an 
imperative in the context of interagency or intergovernmental programs. For example, 
Rosenthal (1984) pointed out, 

Perhaps the most realistic and valuable metaphor for the evaluation of inter- 
governmental programs is that this activity itself became part of a broader 
societal "learning system," rather than a one-shot, clear-cut assessment of 
success or failure. If this metaphor of learning is appropriate, and if key 
public officials can be convinced to take it seriously, then program 
evaluation can become a constructive long-term catalyst for change, (p. 
475) 

The evaluation enterprise in vocational education is centered mostly around outcome 
evaluation. Outcome evaluations are conducted using follow-up studies or by comparing 
specific samples against indicators developed through national surveys. This puts a heavy 
emphasis on indicator development, data collection, and trend analysis. The Education 
Amendments of 1976 (U.S. 94th Congress, 1976) better integrated these three processes; 
yet the evaluation enterprise, as Wentling (1980) noted, is fragmented and lacking an 
institutional anchor. For practical purposes, the targets specified in state plans set the 
parameters of evaluation and also determined what "successful" utilization would be. 
Strickland and Aschc (1987) defined this pattern as "an academic or compliance exercise" 
(p. 13) that has little effect other than raising the budget. 




Apart from outcome evolution, Wending (1980) identified four general concepts for 
developing vocational education evaluation: 

1. Program evaluation is a process rather than a procedure. . . . 
Evaluation is most useful when it is treated as a process — a way of 
decision making — and when it is applied as such. 

2. Program evaluation is more than examining the achievement of 
objectives. Assessment of the extent to which instructional 
programs attain their objectives is not the only dimension of 
program evaluation. The assessment of objectives as they relate to 
the needs of all those involved is of primary importance. 

3. Program evaluation is more than instructional evaluation. 
Instructional evaluation should determine how a career or program 
satisfies the specific needs of the learners; program evaluation is 
more apt to concentrate on how the total program satisfies the 
occupational needs of the community or the training requirements of 
the corporate organization. 

4. Program evaluation is more than evaluating the results of a program. 
Product evaluation involves judging training outcomes and the costs 
incurred for a given program offering, relating these outcomes to, 
prespecified objectives and considering both positive and negative 
monitoring an instructor's performance, weighing the use of 
laboratory materials, or assessing the learning experiences found in 
the training or instructional setting, (pp. 14-16) 



Political context, administrative context, and goal evolution — along with the 
traditional emphasis on evaluator professionalism — all point to the issue of organizational 
learning, which needs to be institutionalized through evaluation designs. On this point 
Schneider (1986) wrote, "Evaluation needs to be viewed as part of an information- 
producing system that feeds into a cyclical policy-making process" (p. 356). 



Learning becomes critical because as Pfeffer and Salancik (1978) argued, 
"Evaluation defines a set of expectations and encourages conformity to those expectations" 
(p. 79). This pressure to conform can be resisted when evaluation is considered as the 
learning mechanism rather than as a verifying mechanism of the planning process, when 
evaluation is conceived as a craft rather than as an applied technology. 



Summary 

In this section, the scope and methodology of evaluation have been shown to 
evolve gradually but primarily in response to evaluation's political or organizational 
context* As the forms of evaluation emerged, evaluation's strategic uses became apparent, 



but its implementation or utilization has been subject to institutional or political context as 
well. Consequently, evaluation utilization has become a critical area of discussion. The 
primary utilization of evaluation will be realized, however, through its capacity to 
institutionalize organizational learning. Changing the thinking and behavior of 
organizations can bring about long-term solutions (National Governors 1 Association, 
1991). 

Hence, our discussion so far has led to the actuality and desirability of the planning- 
evaluation linkage — brought forth by strategic planning on one hand and evaluation 
utilization on the other. To understand this linkage as an integrative process as well as a 
craft activity calls for a shift of metaphor — a shift away from considering planning and 
evaluation as separate proclivities and toward viewing them as aspects of the same 
phenomenon. 



LINKING PLANNING AND EVALUATION 

The need for linkage between planning and evaluation is a major theme in the 
literature. Discussion about whether such linkage actually exists and about how linkage 
can be brought about is sparse and scattered. For example, Wentling (1980) wrote, 
"Integration of evaluation activities such as staff evaluation, resource evaluation, and 
student interest surveys (to name but a few) should facilitate the planning of new programs 
or the planning of changes in existing ones. Seldom, however, is any kind of evaluation 
incorporated in the formulation of program or institutional plans" (p. 24). 

Conceptual Frames of Reference 

To assess the existence of planning and evaluation linkage (or to forge a linkage if 
one does not exist) necessitates a frame of reference that treats planning and evaluation not 
as mutually exclusive processes but as a single process. Frames of reference are important 
in three ways: (1) they give focus on strategic issues, (2) they allow identification of 
instruments, and (3) they provide knowledge of consequences (House, 1983). House 
pointed out that 
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the framing of the social problem depends on the metaphors underlying the 
stories, and how the problems are framed is critical to the solutions that 
emerge. For example, a pervasive description of the social services is that 
they are "fragmented" and the implicit solution to this problem is that they 
be "coordinated." Services seen as "fragmented" could also be seen more 
benignly as "autonomous." Therefore, the underlying metaphor gives 
shape and direction to the problem solution, (p. 8) 

There is an immense conceptual and practical space within which one can frame the 
linkages between planning and evaluation. For example, one can frame them as "the 
linkage between knowledge and organized action" (Millikan, 19S9; Friedmann & Hudson, 
1974). This same framing has been variously noted as "from idea to action" (Alexander, 
1985) and "linking data with action" (Alexander, 1985; Roberts-Gray, Buller, & 
Sparkman, 1987). Alexander wrote, 

This conceptualization of the policy implementation processes views the 
transformation of intent into action as a continuous interactive process. The 
process begins with the first stimulus, which directs attention to a problem, 
goal, or issue. After this stimulus, possible — but not necessary — stages 
linking intention to action are as follows: political mobilization and the 
development of policy, its elaboration in programs (expressed in legislation, 
regulations, plans, and projects), and their implementation, (pp. 41 1-412) 

Evaluation may play a role in each of the stages outlined by Alexander, particularly in the 
identification and definition of the problem to which policy development and 
implementation are directed. As pointed out in the earlier concept paper developed by this 
project, the underlying frame of reference plays an active role in the selective identification 
and interpretation of problems (Asche et al., 1992). 

Another route to linkage in the implementation process was suggested by Roberts- 
Gray et al. (1987): 'Thinking at the beginning about recommendations that may be made at 
the close of the evaluation . . . gives evaluator and program staff a better understanding of 
each other's positions and helps both to see more clearly what needs to be measured and 
why" (p. 681). 

The conceptualization of linkage may thus be conceived primarily in the context of 
policy analysis and policy implementation. There is a convergent trend in the planning 
literature where planning contributes to policy analysis (Alterman & MacRae, 1983). 
Similarly, convergence is found in the evaluation literature where evaluation contributes to 
policy implementation. 
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It is the policy process that makes the linkage real and possible, but the linkage 
between planning and evaluation is not a direct one. Three streams of literature provide the 
basis for understanding linkage: Two streams have been previously discussed. From the 
planning side, there is the strategic planning literature; from the evaluation side, there is the 
evaluation utilization literature. 

Within the administration and policy literature, a third stream deals with the 
organizational learning. A fundamental problem of organization action has been the failure 
to learn from mistakes. Writers such as Crozier (1964), Argyris and Schon (1974), and 
Wildavsky (1979) have distinguished organizational learning from individual members' 
learning. Hedberg (1981) stated, 

Although organizational learning occurs through individuals, it would be a 
mistake to conclude that organizational learning is nothing but the 
cumulative result of their member's learning. Organizations do not have 
brains, but they have cognitive systems and memories. As individuals 
develop their personalities, personal habits, and beliefs over time, 
organizations develop world views and ideologies. Members come and go, 
and leadership changes, but organizations 1 memories preserve certain 
behaviors, mental maps, norms and values over time. (p. 6) 

Organizational learning is a concept that goes beyond the simple formal history of an 
organization. From this perspective, organizational learning can be understood as the 
process of improving actions through better knowledge by identifying and correcting 
errors. Learning, then, is qualitatively different from adaptation, which is often considered 
as strategic action. Hedberg (1981) pointed out that the former involves the understanding 
of reasons beyond the immediate event, the latter simply involves defensive adjustment. 
This perspective is particularly relevant considering the earlier discussion regarding a 
perceived lack of strategic action by vocational education. 

Planning and evaluation are cognitive mechanisms involved in organizational 
learning. Planning gives an organization the foresight to act, while evaluation gives it the 
knowledge to identify the errors in planning. For organizational learning to occur, 
planning and evaluation activities must be linked. The looser the link, the more difficult it 
is for the organization to learn; inaction or ineffective action is the probable result Figure 2 
represents graphically these converging streams in the planning, evaluation, and policy 
literature. 
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Planning as an Aid to Evaluation 



Poister (1986) addressed how the planning process can aid evaluation: 

Planning is the function that can make use of evaluative information to 
develop improvements in such factors as program targeting, objectives- 
resource mix, intervention strategy, program configuration, and service 
delivery arrangements. Solutions for poorly performing programs are not 
necessarily apparent from the results of an evaluation, especially when those 
responsible for program planning do not play roles in formulating 
evaluation designs in the first place* (p. 180) 

Another form of linkage was discussed by Volkema (1983), who identified two 
types of problem formulation heuristics — problem reduction and problem expansion. 
Although problem reduction is the dominant heuristic, experimental results suggest that 
problem expansion has a positive effect on idea generation, particularly for individuals 
working on problems that fall outside their area of expertise. 

In the context of vocational education, Moss (1983) stated that the "professional 
challenge is to make the planning process efficient and as rational as is consistent with our 

democratic values There are, of course, a number of technical planning problems that 

remain to be solved, but more importantly there are critical conceptual issues that demand 
immediate attention" (p. 13). The earlier review of planning theory and practice in 
vocational education indicates a general lack of emphasis by states on strategic planning. 
So far as the present discussion suggests, institutionalizing strategic planning and strategic 
management capacity is one conceptual issue that demands immediate attention. 

Strategic planning capacity can be institutionalized through matrix decision making. 
Waiter and Choate (1984) delineated this as the determination of priorities by contrasting 
"the ability to influence" with "the need for action" (p. 30). Asche et al. (1992) proposed a 
four- way typology of "Source x Nature x Urgency x Feasibility" as a systematic way of 
examining the linking of planning and evaluation. This approach was also suggested as a 
way to examine linking's ultimate effectiveness by looking for disproportionate allocation 
of effort or resources to problems deriving from one or more of the dimensions. 

Strategic planning capacity can be diagnosed by analyzing relevant strategic 
dimensions. Wechsler and Backoff (1986) identified eight such indicators for these 
dimensions: 
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1 

1. 


Strength of external influence 


2. 


Locus of strategic control 


3. 


Impetus for strategic action 


4. 


Strategic orientation 


5. 


Orientation toward change 


6. 


Scope of strategic management 


7. 


Strategic management activity level 


8. 


Direction of strategic movement 



Evaluation as an Aid to Planning 

From the point of view of how the evaluation process can aid planning, Hudson 
(1983) stated, "Evaluation comprises a greater part of the total planning effort. . . . The 
evaluation of particular projects and policies can extend to an evaluation of ends (goal 
setting) as well as means; it can suggest new lines of action (identification of alternatives)" 
(p. 39). Hudson conceived of evaluation, at a minimum, as a subcomponent of planning 
and, ideally as a model of planning. Viewing it as a model reveals certain advantages over 
traditional planning theories: 

First, it [evaluation] begins with specific problems, solutions, and social 
settings, whereas other forms of planning often begin with a nebulous 
attempt at goal setting divorced from any consideration of realistic options. 

Second, [evaluation works with the past and focuses upon the future]. 
Planners, on the other hand, tend to make proposals without concern for the 
lessons of the past. 

Third, evaluation is something in which everyone can become involved 
on some level. It is inherently democratic. ... In contrast, planning 
connotes a more refined expertise — a set of procedures involving 
specialized analysis of data, a creative vision in forecasting the future and 
designing ways to deal with it, a forum of high-level deliberations with 
access to a global overview of ends and means. Evaluation can provide a 
stronger thread of continuity between the intentions embodied in plans and 
the way ordinary people experience their real impacts. 

Fourth, evaluation is characterized by advocacy positions to argue both 
sides of the issue. . . . Evaluation confronts a specific reality, from which 
people emerge able to speak of their experience, and it can propose rival 
hypotheses to focus investigation on critical issues. In contrast, planning 
tends to start with more abstract realities: goal statements, problem 
definitions, a preconceived menu of solutions. 
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Finally, because evaluation is focused on particular options in particular 
settings, it can be efficient in the acquisition and interpretation of data 
needed for its purposes. It uses data on a need-to-know bases. In contrast, 
planning efforts tend to start with a general compilation of all-purpose data, 
without a clear initial idea of which information will in the end made a real 
difference in choosing between one or another course of action, (p. 40) 

Patton (1988) emphasized the integration of evaluation into program delivery as a 
strategy for both increased utility and increased cost effectiveness. In his words, "With 
this approach, evaluation is not an 'add-on* that may be dispensed with as a cost-cutting 

measure; nor is it a luxury to be indulged in with extra dollars The principle then, is 

to work with program staff in articulating a theory of action or program delivery model that 
logically and meaningfully includes data collection points that serve, first and foremost, 
program needs but also evaluation information needs" (p. 89). 

The key contribution of evaluation to planning may lie in focusing policy questions 
and giving them an empirical anchor into the organizational realities. Chelimsky (1984) 
provided a framework that linked the various types of program evaluation and the needs of 
program managers and policymakers for information. The links are forged by evaluative 
information needs. 

In the context of vocational education, Wentling (1980) presented a similar model. 
He wrote, "Conclusions drawn from evaluative findings can be utilized in many ways" (p. 
396). Wentling suggested the Context-Input-Process-Product (CIPP) evaluation model 
developed by Stufflebeam as a way to link planning and evaluation. The interdependent 
information generated in the four types of evaluation (context, input, process, product) 
feeds into the planning process and supports a multiple use perspective. The planning 
process, however, is not the front-end blueprint for implementation action as verified 
through summative evaluation but an emergent consensus over operational directions 
propelled by formative evaluation. 



53 



41 



The Emerging Potential for Indicators 



While authors have addressed the potential contributions of evaluation to planning 
and of planning to evaluation, few have addressed linkage direcdy. However, a recent 
movement, termed "quality indicators, 11 may provide the mechanism by which these 
contributions may be institutionalized. Oakes (1986b) wrote, 

The purposes of indicator systems are to measure the health and 
effectiveness of the education system and help policymakers make better 
decisions. These purposes imply that we select a set of indicators based on 
an understanding of which components of the educational system are critical 
to its health and which features signal important changes in its condition. 
We should also know how the various components of the statistics can be 
used to diagnose current and future conditions, (p. 8) 

Indicators become particularly significant when the policies contain multiple and 
conflicting objectives. Indicators are the instruments by which policy is affected. In 
practical terms, administrative capacity and convenience dictate the use of planning and 
evaluation — not the expertise of the planners or the professionalism of evaluations. Timar 
and Kirp (1987) noted that "schools cannot deliver what they do not control. And since 
they lack control over the most important factors influencing educational outcomes, they 
focus instead or what they can control" (p. 318). Selection of indicators then becomes 
critical for the linkage of planning and evaluation. 

Morell (1979) elaborated how the choice of indicators affects the evaluation 
enterprise. He emphasized that the types of indicators desired vary by level and purpose 
(e.g., classroom teachers versus policymakers). Any given level may share some of the 
concerns of other levels, but there are definite shifts in emphasis. Such shifts effect 
evaluation and, in turn, its usefulness in long-range planning. 

Indicators may serve to circumvent appropriate planning and evaluation procedures. 
Hedrick (1980), for example, pointed out, "If placements become a more central indicator 
for accountability, program administrators at the local level will quickly realize that it is to 
their advantage to run high throughput, short term placement oriented programs for the less 
disadvantaged" (p. 63). In vocational education, for example, Oakes (1986b) pointed out 
that indicators are driven by noneducational factors — factors external to the vocational 
education enterprise itself. Oakes (1986a) summed up the state of vocational education in 
the following terms: "Vocational programs seem to exist in an interesting set of 
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circumstances. Economic and social claims that appear to be largely unsubstantiated drive 
policy, program design, and evaluation. Educational claims that have been attributed to 
participation in vocational programs appear to be largely ignored in policy, practice, and 
research" (p. 39). Asche (1989) stated, 

To the extent that we are not clear as to what vocational education should 
accomplish, we are left in a position of having externally imposed indicators 
and performance standards dictate vocational education's purposes and 
activities. The indicators selected for vocational education must be those 
which are linked to attainable and worthwhile goals or we will be in a 
position of using our resources to accomplish someone else's agenda, (p. 
5) 



Constraints on Linkage 

The constraints on effective linkage are more than conceptual. Constraints exist in 
the administrative capacity through which planning and evaluation take place. For 
example, if budget allocations are made to reinforce the optimality of the planning and 
evaluation process, then budgetary time tables must allow informed policy debate. If 
evaluation research is to provide feedback for policy planning in a timely fashion, then its 
timing is critical for program selection, survey design, and utilization planning. 

One way in which linkages can be effective is by the expertise of what Oakes 
(1986a) termed a "critical methodology." She stated, 

Critical methodology requires collaboration. Subjection of knowledge to 
continuous critical reflection demands that participants attend to how 
educational structures, content, processes, values, and beliefs — including 
the inquiry process itself — are linked to the social and political forces in the 
setting and to the large social, political, and economic context in which it is 
situated, (pp. 44-45) 

For vocational education, Strickland and Asche (1987) identified three 
organizational factors that may serve to constrain effective utilization of evaluation. The 
three factors arc the regulative nature of multilevel funding, the discrete service or program 
areas in vocational education, and the diversity of local administrative structures. To these 
three we add the multiplicity of agents, contending perspectives and ideologies, and mixed 
policy messages. 



Hoachlander (1982) noted that "many of the shortcomings of the vocational 
education act are not attributable to Excessive' federal regulation, but rather to poorly 
conceived regulation and, in some instance, the outright failure to regulate at all" (p. 440). 
The reason planning and evaluation linkage is problematic concerns what Benson and 
Lareau (1982) referred to as the uneasy place of vocational education — that is, in co- 
existence with the academic track within the comprehensive school systems. 

Given the context of vocational education, it is important to find out whether there 
is congruence among the broad policy goals; the specific regulations created to achieve 
those goals; and ultimately, the programs implemented at the client level. As Timar and 
Kirp (1987) asked, "Is there a good fit between the ends (substantive rationality) and the 
means (formal rationality) used to achieve those ends?" (p. 310). The matching of 
expectations and outcomes in vocational education was discussed by Lotto (1986) who 
reported that there is a serious mismatch between them. 

The procedural actions that aid the linkage between plaryiing and evaluation can be 
variously conceived as disjointed incrementalism (Lindblom, 1959), participative decision 
making (Friedmann, 1973), or pragmatism (Hoch, 1984). Blanton and Alley (1978) 
proposed some useful procedures for such linkage. Their focus was on the integration of 
feedback and the cognitive and institutional obstacles to such integration. They discussed 
two major obstacles: (1) blocks to hearing, attending to, or understanding feedback; and 
(2) effective difficulties manifested in emotional and ideological resistance to new ideas. 
Blanton and Alley suggested procedures for improving "hearing" — including cultivating 
different styles of presentation (visual displays and charts can be useful to supplement the 
narrative, as well as the timing of presentations), providing audience relevance (reporting 
separately to each stakeholder group), and articulating interpretive skills (command over 
quantitative and qualitative arguments). The suggested techniques for overcoming 
emotional and ideological resistance included increasing the credibility of feedback 
(focusing more on the use of information than on being judgmental), avoiding polarization 
between planning and evaluation staff (utilizing matrix decision making), dealing with 
ideological resistance (increasing the involvement of stakeholders), and pacing findings 
(gradually collecting and diffusing information and ideas). 

One methodological approach that increases the possibility of linkage is what Guba 
and Lincoln (1987) proposed as the "naturalistic method." This method provides a way to 
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deal with values conflicts that calls for full collaboration with problem stakeholders. Full 
collaboration means that stakeholders are given the opportunity to provide input at every 
stage of evaluation. Evaluation thus becomes a part of the goal-setting or planning process. 



The administrative arrangements that facilitate linkage will vary according to local 
historical and political conditions as well as the nature of the relationship established with 
federal agencies. Polivka and Stryker (1983) identified three important mechanisms: 

1 • The budget development process — review each budget request for 
program areas where an evaluation has been conducted. This 
enables the agency to determine the extent to which the request 
reflects the recommendations generated by the evaluation. 

2. Performance agreement monitoring — preparation of annual 
performance agreement with the planning and budget office, 

3. Program budget measures — inclusion of process and outcome 
variables either used in the conduct of evaluations or developed from 
evaluation findings in biennial budget recommendations, (p. 257) 



A New Metaphor for Linkage 

This section began with the need for a new metaphor that relates planning and 
evaluation as mutually inclusive processes rather than inherently exclusive domains. Hoch 
(1984) provided the needed metaphor "doing good and being right" Although developed 
for the planning process, the metaphor has sufficient power to include the evaluation 
process. The metaphor captures the pragmatic basis of both planning and evaluation. 
Thus, pragmatism allows planning and evaluation to be linked. Under this connection, 
both planning and evaluation become a "process of practical inquiry in which problematic 
experience stimulates the critical assessment of alternative solutions in a context of 
democratic participation and debate" (Hoch, 1984, p. 340). 

This pragmatic connection between planning and evaluation in vocational program 
administration can be achieved by institutionalized strategic planning, strategic 
management, evaluation utilization, and organization learning. The connection is not a 
direct bonding of the processes but one that is conceptually and practically made possible in 
the policy implementation process. It is this implementation orientation that encourages 
linkages between planning and evaluation activities; it does not keep them as two entities, 
an orientation which often requires external expertise. This institutionalized linkage is 
presented graphically in Figure 3. 
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Before embracing pragmatism fully, one needs to consider that the capacity to act 
and then think pragmatically is also circumscribed by historical context, institutional forms, 
and cultural bias. In this respect, Oakes' (1986a) proposal for critical reflection and Guba 
and Lincoln's (1987) proposal for naturalistic inquiry are two important qualifications that 
need to be incorporated in making the pragmatic connection. 



Summary 

The present review has emphasized the importance of linking planning and 
evaluation in vocational education. It has portrayed such linking as taking place in the 
larger context of policy design and policy implementation. The policy design and 
implementation process was seen as continuous and cyclical with planning and evaluation 
facilitating congruence through several mechanisms. 

First, a metaphor of conceptual as well as operational linking of planning and 
evaluation was proposed. The research streams of strategic planning and management, 
evaluation utilization, and organizational learning provided the empirical basis for 
conceptual linking. Several recommendations were derived from the literature, including 
use of naturalistic methods, application of critical methodology in policy debate, use of 
quality or performance indicators, structure of the budget process, and methods of 
presentation of data. Many of the recommended procedures in the evaluation utilization 
literature and many of the proposals for more participative planning approaches (e.g., 
transactive planning) were equally relevant to the linking process. 

Most of what has been reported in this review is theoretical. While many of the 
findings derive from empirical study of organizations, little or no research on their 
application to the complex milieu of vocational education exists. A third report prepared for 
this project presents the results of just such an inquiry using both traditional survey 
methods and in-depth case studies of exemplary state programs (Elson, Oliver, & 
Strickland, 1992). 




REFERENCES 



Alexander, E. R. (1985). From idea to action. Administration and Society, 16(4), 403- 
425. 

Alterman, R., & MacRae, D., Jr. (1983). Planning and policy analysis— Converging or 
diverging trends? Journal of the American Planning Association, 49(2), 200-215. 

Argyris, C, & Schon, D. (1974). Theory in practice: Increasing professional 
effectiveness. San Francisco, CA: Jossey-Bass. 

Asche,F. M. (1985). Planning for secondary vocational education in Georgia. Atlanta: 
Georgia State Department of Education, Office of Vocational Education. 

Asche, F. M. (1989). Standards and measures of performance: Indicators of quality for 
Virginia vocational education programs. Blacksburg: Virginia Polytechnic Institute 
and State University. 

Asche, F. M., Strickland, D. C, & Elson, D. E. (1992). Linking planning and evaluation 
in vocational education: Issues and concepts. Berkeley: National Center for 
Research in Vocational Education, University of California at Berkeley. 

Barlow, M. L. (1965). The challenge to vocational education. In M. Barlow (Ed.), 
Vocational education (64th yearbook of the National Society for the Study of 
Education) (pp. 1-18). Chicago, EL: University of Chicago Press. 

Barry, B., & Rae, D. (1975). Political evaluation. In F. I. Greenstein & N. Polsky 
(Eds.), Handbook of Political Science (Vol. 1) (pp. 339-348). Reading, MA: 
Addison-Wesley. 

Beaulieu, R. A., & Erickson, S. R. (1977). An information network and simulation 
model for vocational education (Final Report, Rhode Island Model II, Vol. 1). 
Providence, RI: State Department of Education. (ERIC Document Reproduction 
Service No. ED 138 820) 



Beebc, C. E. (1966). The quality of education in developing countries. Cambridge, MA: 
Harvard University Press. 

Benson, C. S., & Lareau, A. P. (1982). The uneasy place of vocational education. 
Education and Urban Society, 75(1), 104-124. 

Blanton, J., & Alley, S. (1978). How evaluation findings can be integrated into program 
decision making. Community Mental Health Journal, 14(3), 345-353. 

Boruch, R. F., & Cordray, D. S. (1980). An appraisal of educational program 
evaluations: Federal, state, and local agencies. Evanston, IL: Northwestern 
University. 

Braybrooke, D., & Lindblom, C. E. (1983). A strategy of decision: Policy evaluation as 
a social process. New York, NY: Free Press. 

Brophy, K., Groteleuschen, A., & Gooler, D. (1974). A blueprint for program 
evaluation (Occupational Paper No. 1). Champaign-Urbana: University of Illinois, 
College of Education, Office of Professional Services. 

Browne, A., & Wildavsky, A. (1987). What should evaluation mean to implementation? 
In D. Palumbo (Ed.), The politics of program evaluation. Sage yearbooks in 
politics and public policy (Vol. 15K(PP- 146-172). Beverly Hills, CA: Sage 
Publications. 

Campbell, D. T., & Stanley, J. C (1963). Experimental and quasi-experimental designs 
for research. Chicago, IL: RandMcNally. 

Chelimsky.E. (1983). Program evaluation and appropriate governmental change. InC. 
E. Gilbert (Ed.), Implementing governmental change. The annals of the American 
Academy of Political and Social Science (Vol. 466) (pp. 103-1 18). Beverly Hills, 
CA: Sage Publications. 

Chelimsky.E. (1984). Linking program evaluation to policy and management information 
needs. Optimum, 15(4), 72-99. 



Cingranelli, D. L., Hofferbert, R. I., & Ziegenhagen, E. A. (1980). Goal evolution 
through implementation: The problem for policy evaluation. Policy Studies 
Journal, 8(3), 1229-1238. 

Copa, G. H., & Moss, J., Jr. (Eds.). (1983). Planning and vocational education. New 
York, NY: McGraw-Hill. 

Crozier, M. (1964). The bureaucratic phenomenon. Chicago, IL: University of Chicago 
Press. 

Daniels, M M & Wirth, C. J. (1983). Paradigms of evaluation research: The development 
of an important policy making component. American Review of Public 
Administration, 77(1), 33-44. 

David, H. (1981). The vocational education study: The final report (Publication No. 8). 
Washington, DC: U.S. Office of Education. 

Davidoff, P. (1966). Advocacy and pluralism in planning. Journal of the American 
Institute of Planners, 3/(4), 331-338. 

DeLeon, P. (1983). Policy evaluation and program termination. Policy Studies Review, 
2(4), 631-643. 

Elson, D. E., Oliver, J. D., & Strickland, D. C. (1992). Strategies for linking planning 
and evaluation in vocational and technical education. Berkeley: National Center for 
Research in Vocational Education, University of California at Berkeley. 

Etzioni, A. (1967). Mixed scanning: A "third" approach to decision making* Public 
Administration Review, 27(6), 387-391. 

Franchak, S. J. (1981). Using evaluation results: Guidelines and practices for using 
vocational evaluation effectively. Columbus: National Center for Research in 
Vocational Education, Ohio State University. 

Friedmann, J. (1973). Retracking America: A theory of transactive planning. Garden 
City, NY: Anchor Books. 



51 



63 



Fricdmann J., & Hudson, B. (1974). Knowledge and action: A guide to planning theory. 
Journal of the American Institute of Planning, 4(1), 3-16. 



Grabow, S., & Heskin, A. (1973). Foundations for a radical concept of planning. 
Journal of the American Institute of Planning, 39(2), 106- 1 14. 

Guba, E., & Lincoln, Y. (1987). The countenances of fourth-generation evaluation: 
Description, judgment and negotiation. In D. Palumbo (Ed.), The politics of 
program evaluation (pp. 202-234). Beverly Hills, CA: Sage Publications. 

Hedberg, B. (1981). How organizations learn and unlearn? In P. C. Nystrom & W. H. 
Starbuck (Eds.), Handbook of organizational design (pp. 3-27)* London, England: 
Oxford University Press. 

Hedrick, T. E. (1980). Interactions among evaluation, program implementation, and 
policy in a decentralized system. Policy Studies Journal, 8(3), 63-69. 

Hoachlander, E. G. (1982). Considering deregulation in education: Experience with the 
vocational education act Education and Urban Society, 14(4), 425-44L 

Hoch, C. (1984). Doing good and being right, the pragmatic connection in planning 
theory. Journal of the American Planning Association, 50(3), 335-345. 

House, E. R. (1983). How we think about evaluation. In E. R. House (Ed.), Philosophy 
of evaluation. New directions for program evaluation (Vol. 19) (pp, 5-25). San 
Francisco, CA: Jossey-Bass. 

Hudson, B., & Davis, R. G. (1976). Knowledge networks for educational planning. 
Los Angeles: University of California at Los Angeles, School of Architecture and 
Urban Planning. 

Hudson, B. M. (1983). Planning: Typologies, applications, issues, and contexts. In G. 
H. Copa & J. Moss, Jr. (Eds.), Planning and vocational education (pp. 18-44). 
New York, NY: McGraw-Hill. 



52 



Koontz, H. (1963). The rationale for planning. In D. M. Bowman & F. M. Fillerman 
(Eds.), Management organization and planning (pp. 65-85). New York, NY: 
McGraw-Hill. 

Kuhn, T. (1962). The structure of scientific revolutions. Chicago, IL: University of 
Chicago Press. 

Leviton, L. C, & Hughes, E. F. (1981). Research on the utilization of evaluations. 
Education Review, 5(4), 525-548. 

Light, R. J. (1980). Evaluation methodology for education programs with diffuse goals. 
Education and Urban Society, 73(1), 109-136. 

Lindblom, C. E. (1959). The science of muddling through. Public Administration 
Review, 79(1), 79-88. 

Lotto, L. S. (1986). Expectations and outcomes of vocational education: Match or 
mismatch. Journal of Vocational Education Research, 77 (1), 41-60. 

Mannheim, K. (1949). Ideology and Utopia. New York, NY: Harcourt. 

Maynard-Moody, S. (1983). Program evaluation and administrative control. Policy 
Studies Review, 2(3), 368-384. 

McDonnell, L. M., & Grubb, W. N. (1991). Education and training for work: The 
policy instruments and the institutions. Berkeley: National Center for Research in 
Vocational Education, University of California at Berkeley. 

Mettsner, A. (1972). Political feasibility and policy analysis. Public Administration 
Review, 32(6), 859-867. 



Millikan, M. F. (1959). Inquiry and policy: The relation of knowledge to action. In D. 
Lerner (Ed.), The human meaning of the social sciences (pp. 158-180). Cleveland, 
OH: World Publishing. 



Mintzberg. H. (1983). Power in and around organizations. Englcwood Cliffs, NJ: 
Prentice-Hall. 

Morell,J.A. (1979). Program evaluation in social research. New York, NY: Pergamon. 

Morgan, J. M., & Cohn, E. (1977). Resource allocation within secondary schools: A 
goal programming approach. In American Statistical Association (Ed.), 
Proceedings of the Social Statistics Section, American Statistical Association (pp. 
646-652). Washington, DC: American Statistical Association. 

Moss, J., Jr. (1983). State-level planning in vocational education. In G. H. Copa & J. 
Moss, Jr. (Eds.), Planning and vocational education (pp. 7-17). New York, NY: 
McGraw-Hill. 

Nachmias,D. (1980). The role of evaluation in public policy. Policy Studies Journal, 
5(3), 1163-1172. 

National Governors 1 Association. (1991). Excellence at work: A state action agenda. 
Washington, DC: Author. 

Oakes,J. (1986a). Beneath the bottom line: A critique of vocational education research. 
Journal of Vocational Education Research, ii(2), 33-49. 

Oakes,J. (1986b). Educational indicators: A guide for policy makers. Washington, DC: 
RAND Corporation, Center for Policy Research in Education. 

Orcutt, G. H. (1961). Microanalysis of socioeconomic systems: A simulation study. 
New York, NY: Harper and Row. 

O'Reilly, P. A. (1975). The state planning process in vocational education (Project 
Baseline Supplemental Report)* Flagstaff: Northern Arizona University. 

Palumbo, D. J. (1987). Politics and evaluation. In D. Palumbo (Ed.), The politics of 
program evaluation (pp. 12-46), Beverly Hills, CA: Sage Publications. 



54 




86 



Palumbo, D. L, & Nachmias, D. (1983). The preconditions for successful evaluation: Is 
there an ideal paradigm? Policy Sciences, 16(1), 77-83. 

Patton, M.Q. (1982). Practical evaluation. Beverly Hills, CA: Sage Publications. 

Patton, M.Q. (1986). Utilization-focused evaluation (2nd ed.). Beverly Hills, CA: Sage 
Publications. 

Patton, M. Q. (1987). Evaluation's political inherency: Practical implications for design 
and use. In D. Palumbo (Ed.), The politics of program evaluation (pp. 142-158). 
Beverly Hills, CA: Sage Publications. 

Patton, M. Q. (1988). Integrating evaluation into a program for increased utility and cost- 
effectiveness. In J. A. McLaughlin, L. J. Weber, & R. W. Covert (Eds.), 
Evaluation utilization. New directions for program evaluation (Vol. 39) (pp. 85- 
94). San Francisco, CA: Jossey-Bass. / 

Peattie, L. (1968). Reflections on advocacy planning. Journal of the American Institute 
of Planners, 34(2), 80-88. 

Perkins, D. N. T. (1977). Evaluating social intervention: A conceptual schema. 
Evaluation Quarterly, 1(4), 639-656. 

Pfeffer, J., & Salancik, G. (1978). The external control of organizations. New York, 
NY: Harper and Row. 

Poister, T. H. (1986). Linking program planning, evaluation, and management: Will it 
ever happen? Public Administration Review, 46(2), 179-183. 

Polivka, L., & Stryker, L. T. (1983). Program evaluation and policy process in state 
government: An effective linkage. Public Administration Review, 43(3), 255-259. 

Pressman, J., & Wildavsky, A. (1979). Implementation. Berkeley: University of 
California Press. 



55 



Rich, R. F. (1977). Uses of social science information by federal bureaucrats: 
Knowledge for action versus knowledge for understanding. In C. H. Weiss (Ed.), 
Using social research in public policy making (pp. 199-211). Lexington, MA: 
Lexington Books. 

Roberts-Gray, C. R., Buller, A., & Sparkman, A. (1987). Linking data with action — 
Procedures for developing recommendations. Evaluation Review, 11(5), 678-683. 

Rosenthal, S. R. (1984). New directions for evaluating intergovernmental programs. 
Public Administration Review, 44(6), 469-475. 

Schmiddein, F. A. (1983). Comprehensive and incremental decision paradigms and their 
implications for educational planning. In G. H. Copa & Jc Moss, Jr. (Eds.), 
Planning and vocational education (pp. 48-80). New York, NY: McGraw-Hill. 

Schneider, A. N. (1986). The evaluation of a policy orientation for evaluation research: A 
guide to practice. Public Administration Review, 46(4), 356-363. 

Scriven, M. (1973). The methodology of evaluation. In B. R. Worthen & J. R. Sanders 
(Eds.), Educational evaluation: Theory and practice (pp. 60-106). Belmont, CA: 
Wadsworth. 

Simmons, J. (1974). Education, poverty and development (Working Paper No. 188). 
Washington, DC: World Bank. 

Simon, H. A. (1945). Administrative behavior: A study of decision making processes in 
administrative organization. New York, NY: Free Press. 

Smith, M. F. (1988). Evaluation utilization revisited. In J. A. McLaughlin, L. J. Weber, 
& R. W. Covert (Eds.), Evaluation utilization. New directions for program 
evaluation (Vol. 39) (pp. 7-19). San Francisco, CA: Jossey-Bass. 

Strickland, D. C, & Asche, M. (1987). Enhancing utilization: A proposal for a modified 
utilization focused model for vocational education evaluation. Journal of Vocational 
Education Research, 12(4), 13-34. 



ERLC 



56 66 



Swanson, J. C. (1966). Development of federal legislation for vocational education. 
Chicago, IL: American Technical Society. 

Thompson, J. F. (1973). Foundations of vocational education: Social and philosophical 
concepts. Englewood Cliffs, NJ: Prentice-Hall. 

Timar, T. B., & Kirp, D. L. (1987). Educational reform and institutional competence. 
Harvard Educational Review, 57(3), 308-330. 

U.S. 64th Congress. (1917). Smith-Hughes Act. Washington, DC: Government 
Printing Office. 

U.S. 88th Congress. (1963). Vocational Education Act of 1963. Washington, DC: 
Government Printing Office. 

U.S. 90th Congress. (1968). Vocational Education Amendments of 1968. Washington, 
DC: Government Printing Office. 

U.S. 92nd Congress. (1972). Education Amendments of 1972. Washington, DC: 
Government Printing Office. 

U.S. 94th Congress. (1976). Education Amendments of 1976. Washington, DC: 
Government Printing Office. 

U.S. 98th Congress. (1984). Carl D. Perkins Vocational Education Act. Washington, 
DC: Government Printing Office. 

U.S. 101st Congress. (1990). Carl D. Perkins Vocational and Applied Technology 
Education Act of 1990. Washington, DC: Government Printing Office. 

Volkema, R. J. (1983). Problem formulation in planning and design. Management 
Science, 29(6), 639-652. 

Walter, S., & Choate, P. (1984). Thinking strategically—A primer for public leaders. 
Washington, DC: Council of State Planning Agencies. 



57 f?S 



Wechsler, B., & Backoff, R. W. (1986). Policy making and administration in state 
agencies: Strategic management approaches. Public Administration Review, 
46(4), 321-327. 

Weiss, C. H. (1988). If program decisions hinged only on information: A response to 
Patton. Evaluation Practice, 9(3), 15-28. 

Weiss, C. H., & Bucuvalas, M. J. (1977). Social science research and decision making. 
New York, NY: Columbia University Press. 

Wentling, T. L. (1980). Evaluating occupational education and training programs (2nd. 
ed.). Boston, MA: Allyn& Bacon. 

Wildavsky, A. (1979). Speaking truth to power: The art and craft of policy analysis. 
Boston, MA: Little, Brown. 

Wirth,A. G. (1983). Productive work in industry and schools: Becoming persons again. 
Lanham, MD: University Press of America. 

Worthen, B. R., & Sanders, J. R. (1987). Educational evaluation: Alternative 
approaches and practical guidelines. White Plains, NY: Longman. 



ERLC 



58 



